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educational process at the Reedsburg campus of Madison Area Technical College 
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each (students entering MATC ' s associate degree nursing program in August 
1995 and August 1996 and graduating in May 1997 and 1998, respectively) . Each 
cohort was selected through a randomized selection process from the first-day 
applicants who fulfilled all the program's prerequisites for entry. The 
students' critical thinking skills were examined by having them complete the 
California Critical Thinking Skills Test (CCTST) before starting and after 
completing their program of study. The study confiinned CCTST pre- and 
posttesting to be a viable means of assessing changes in students' critical 
thinking skills. One cohort exhibited a significant increase in critical 
thinking. Both cohorts manifested significant gains in their evaluation and 
inductive reasoning skills; however, neither cohort demonstrated significant 
gains in their analysis, inference, and deductive thinking skills. (The 
report contains 45 references and 17 tables/figures. Appended are a 
discussion of validation of the California Critical Thinking Skills Test and 
the agreement to participate as a research subject.) (MN) 
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Critical thinking is a major component of nursing education. Programs in 

nursing must assess critical thinking as a required outcome for national accreditation by 

the National League for Nursing. In March, 1996, Madison Area Technical College 

Associate Degree Nursing faculty identified critical thinking as a major content thread to 

be carried as a priority for instruction through all four semesters of the Associate Degree 

Nursing program. 

The purpose of this study was to determine the degree of change in critical 
thinking skills in Associate Degree Nursing students as they progress through their 
educational process at Madison Area Technical College. Cohorts of twenty-four 
students, entering the Associate Degree Nursing program at Madison Area Technical 
College - Reedsburg campus, in August, 1995 and August, 1996 and graduating in May, 
1997 and May, 1998, comprised the samples. The California Critical Thinking Skills 
Test - Form A (CCTST) was administered to the entering Associate Degree Nursing class 
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in August of 1995 and in August of 1996 - prior to the beginning of program instruction. 
The California Critical Thinking Skills Test Form A was also administered at the 
completion of the program of study (May, 1997 and May, 1998). The California Critical 
Thinking Skills Test - Form A was administered by the researcher at the Madison Area 
Technical College - Reedsburg campus. 

To assess the degree of change in thinking skills, the pre-test CCTST scores were 
compared to the post-test CCTST scores. The comparison was made by subtracting the 
pre-test score from the post-test score for each individual. The resulting change scores 
were analyzed with the t-test of differences. 

The major outcome of this study was to identify CCTST pre- and post-testing of 
MATC’s ADN students as a viable means of assessing their change in critical thinking 
skills. One group showed a significant increase in the total score for the CCTST, based 
on positive gains in their scores for the five sub-tests that impacted the total score. 
Further outcomes were to identify a significant change in both evaluation and inductive 
reasoning thinking skills in the two groups of students: One group showed a significant 
increase in evaluation thinking skills; the other group had a positive gain in evaluation 
thinking skills, although it was not statistically significant. One group showed a 
significant increase in the critical thinking skill subset Inductive Reasoning. An 
additional benefit of the study was to identify limited, if any, change in analysis, 
inference, and deductive thinking skills in the program students. These areas of limited 
change then become areas to be further evaluated by the ADN program. 
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Chapter 1 



Research Problem and Objectives 



Introduction 

Critical thinking is a major component of nursing education. The practice of 
professional nursing necessitates the application of critical thinking skills to problem 
solve, develop ideas, and provide quality health care. In nursing and other health care 
professions, life or death may depend on the problem-solving done in an emergency 
situation or upon the interpretations of subtle changes in a recovering patient’s status. 
Critical thinking is an essential skill (Facione, Facione, & Giancario, 1998). In 1997, the 
National League for Nursing, the educational accreditation agency for nursing, identified 
critical thinking as one of four performance indicators for national accreditation. These 
performance indicators are to be assessed to provide evidence to demonstrate the 
purposes of the educational program and the program effectiveness. Therefore, 
programs in nursing must assess critical thinking as a required outcome for national 
accreditation (National League for Nursing, 5/29/98). 

Madison Area Technical College has a two year Associate Degree Nursing 
program, offered at four of its campuses: Truax, Reedsburg, Watertown, and Fort 
Atkinson. The program curriculum is the same throughout the district. On March 1, 
1996, nursing faculty identified critical thinking as a major content thread to be carried 
as a priority for instruction through all four semesters of the Associate Degree Nursing 
program (ADN Minutes). 



Research Problem 



The purpose of this study is to determine the degree of change in critical thinking 
skills in Associate Degree Nursing students as they progress through their educational 
process at Madison Area Technical College. 

Cohorts of twenty-four students, entering the Associate Degree Nursing program 
at Madison Area Technical College - Reedsburg campus, in August, 1995 and August, 
1996 and graduating in May, 1997 and May, 1998, comprised the sample. The California 
Critical Thinking Skills Test - Form A was administered to the entering Associate Degree 
Nursing class in August of 1995 and in August of 1996 - prior to the beginning of 
program instruction. The California Critical Thinking Skills Test Form A was also 
administered at the completion of the program of study (May, 1997 and May, 1998). The 
California Critical Thinking Skills Test - Form A was administered by the researcher at 
the Madison Area Technical College - Reedsburg campus. 

Research Hypotheses 

The following hypotheses were tested; 

1 . There will be an increase in the critical thinking skills of Associate 
Degree Nursing students from entry to completion of the Associate 
Degree Nursing program at Madison Area Technical College - Reedsburg. 

2. There will be an increase in the critical thinking skill subset “Analysis” in 
Associate Degree Nursing students from entry to completion of the 
Associate Degree Nursing program at Madison Area Technical College - 
Reedsburg. 
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3. There will be an increase in the critical thinking skill subset “Evaluation” 
in Associate Degree Nursing students from entry to completion of the 
Associate Degree Nursing program at Madison Area Technical College- 
Reedsburg. 

4. There will be an increase in the critical thinking skill subset “Inference” 
in Associate Degree Nursing students from entry to completion of the 
Associate Degree Nursing program at Madison Area Technical College - 
Reedsburg. 

5. There will be an increase in the critical thinking skill subset “Deductive 
Reasoning” in Associate Degree Nursing students from entry to 
completion of the Associate Degree Nursing program at Madison Area 
Technical College- Reedsburg. 

6. There will be an increase in the critical thinking skill subset “Inductive 
Reasoning” in Associate Degree Nursing students from entry to 
completion of the Associate Degree Nursing program at Madison Area 
Technical College- Reedsburg. 

Research Objectives 

The objectives of this study were; 

1 . To determine the degree of change in critical thinking skills in Associate 
Degree Nursing students at Madison Area Technical College - Reedsburg. 

2. To assess the feasibility of using the California Critical Thinking Skills 
Test Form A to develop performance standards related to critical thinking 
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for the Madison Area Technical College Associate Degree Nursing 
program. 

Need for the Study 

All levels of nurses are expected to embrace and practice critical thinking in their 
professional roles. Although Madison Area Technical College faculty have incorporated 
critical thinking as a thread through all four semesters of the program and in each nursing 
course, performance standards for critical thinking have not been identified for the 
program. A consistent assessment plan for critical thinking skills in the Associate 
Degree Nursing students needs to be delineated for successful continuance of the 
Madison Area Technical College Associate Degree Nursing program in order to maintain 
its accreditation status and to provide quality graduate nurses. An assessment 
methodology for critical thinking needs to be set in place. 

Limitations 

The scope of this study is limited to two cohorts of Associate Degree Nursing 
students, at the Madison Area Technical College - Reedsburg campus, entering August, 
1995 and graduating in May, 1997, and entering August, 1996 and graduating in May, 
1998. Only students who were enrolled with the program for two consecutive years were 
included in the study. Students admitted with advanced standing or transferring into the 
program from another technical college district or MATC campus were not included in 
the study. 

Assumptions 

It is assumed that the Associate Degree Nursing students at the Madison Area 
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Technical College - Reedsburg campus are representative of the population for the entire 
Madison Area Technical College Associate Degree Nursing program, located at 
Madison, Watertown, Fort Atkinson, and Reedsburg. Madison Area Technical College - 
Reedsburg Associate Degree Nursing graduates consistently comprise 23% (averaging 22 
graduates out of approximately 95 graduates) of the total number of Associate Degree 
Nursing program graduates. Associate Degree Nursing students enrolled at the Madison 
Area Technical College - Reedsburg campus have the same entrance requirements and 
pre-requisites for admission to the program as students at other campuses. Graduates of 
the Madison Area Technical College - Reedsburg Associate Degree Program have the 
same pass rate on the registered nurse licensure exam as those Associate Degree Nursing 
graduates from other Madison Area Technical College campuses (averaging 95% over 
ten years). 

Definition of Terms 

In 1990, a consensus definition of critical thinking skills emerged as a Delphi 
study commissioned by the American Philosophical Association to clarify the 
conceptualization of critical thinking. These consensus definitions are described in 
Cntip sl Thinking; A Statement of Expert Consensus for Purposes of Educational 
Assessment and Instructioa The Delphi Report, (Facione, 1990); 

Analysis: To identify the intended and actual inferential relationships among statements, 
questions, concepts, descriptions or other forms of representation intended to express 
beliefs, judgments, experiences, reasons, information, or opinions, which includes the 
sub-skills of examining ideas, detecting arguments, and analyzing arguments into their 
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component elements, (p. 8) 

Critical Thinking: Purposeful, self-regulatory judgement which results in 
interpretation, analysis, evaluation, and inference, as well as explanation of the 
evidential, conceptual, methodological, criteriological, or contextual considerations upon 
which judgment is based, (p. 2) 

Deductive Reasoning: The assumed truth of the premises purportedly necessitates the 
truth of conclusion. Traditional syllogisms, algebraic, geometric, and set-theoretical 
proofs in mathematics are examples of deductive reasoning, (p. 5) 

Evaluation: To assess the credibility of statements or other representations which are 
accounts or descriptions of a person’s perception, experience, situation, judgment, belief, 
or opinion, which includes the sub-skills of assessing claims and assessing arguments. 
Evaluation also means to state the results of one’s reasoning; to justify that reasoning in 
terms of the evidential, conceptual, methodological, criteriological, and contextual 
considerations upon which one s results were based; and to present one’s reasoning in 
the form of cogent arguments, which includes the sub-skills of stating results, justifying 
procedures, and presenting arguments, (p. 9) 

Explanation: To state the results of one’s reasoning; to justify one’s reasoning in terms 
of the evidential, conceptual, methodological, criteriological, and contextual 
considerations upon which one’s results were based; and to present one’s reasoning in 
the form of cogent arguments, which includes the sub-skills of stating results, justifying 
procedures, and presenting arguments, (p. 11) 

Inference: To identify and secure elements needed to draw reasonable conclusions; to 

O 

ERIC 



14 



7 



form conjecture and hypotheses, to consider relevant information and to educe the 
consequences flowing from data, statements, principles, evidence, judgments, beliefs, 
opinions, concepts, descriptions, questions, or other forms of representation, which 
includes the sub-skills of querying evidence, conjecturing alternatives, and drawing 
conclusions, (p. 10) 

Interpretation: To comprehend and express the meaning or significance of a wide 
variety of experiences, situations, data, events, judgments, conventions, beliefs, 
procedures or criteria, (p. 7) 

Inductive Reasoning: An argument’s conclusion is purportedly warranted, but not 
necessitated, by the assumed truth df its premises. Scientific confirmation and 
experimental confirmation are examples of inductive reasoning, (p. 5) 

Self-Regulation: Self-consciously to monitor one’s cognitive activities, the elements 
used in those activities, and the results educed, particularly by applying skills in analysis 
and evaluation to one’s own inferential judgments with a view toward questioning, 
confirming, validating, or correcting either one’s reasoning or one’s results, which 
includes the sub-skills of self-examination and self-correction, (p. 12) 





Chapter II 

Review of Related Literature 



Introduction 

The literature review was conducted through ERIC and electronic search banks 
and library holdings at Madison Area Technical College, the University of Wisconsin - 
Stout, the University of Wisconsin - Madison, the Wisconsin Technical College System, 
and through Internet search banks. For the purpose of this study, the literature review 
was focused upon critical thinking, assessment of critical thinking, critical thinking in 
nursing, assessment of critical thinking in nursing, and critical thinking at Madison Area 
Techmcal College and its Associate Degree Nursing program. 

Critical Thinking 

The concern of philosophers with critical thinking dates back to ancient times, 
when Plato and Aristotle were interested in formal logic systems. Early Hindu writings 
showed some parallel development in the analysis of concepts (Miller & Connelly, 

1996 ). 

For the better part of the twentieth century, schooling has concerned itself with 
developing techniques, skills, and factual knowledge, while training has concerned itself 
with supplying scripted solutions to predetermined solutions to predetermined problems. 
However, education as practiced in the better liberal arts based professional programs, is 
structured to include the goal of teaching people to think (Facione, Facione, & Giancario, 
1998 ). 

O 
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In this century, John Dewey is recognized as the modern-day founder of the 
critical thinking movement (Ennis, 1993). Dewey in How We Think stated: 

...thinking enables us to direct our activities with foresight and to plan according 
to ends-in-view, or purposes of which we are aware. It enables us to act in 
deliberate and intentional fashion to attain future objects or to come into 
command of what is now distant and lacking. By putting the consequences of 
different ways and lines of action before the mind, it enables us to know what we 
are about when we act. It converts action that is merely appetitive, blind, and 
impulsive into intelligent action. (1910, p. 17) 

In 1941, Edward Glaser defined critical thinking as dependent both on one’s 
notion of what it means to think critically and on one’s view of the reasons for critical 
thought. He stated that critical thinking involved three principal elements, “Attitude of 
being disposed to consider in a thoughtful, perceptive marmer the problems and subjects 
that come within the range of one’s experience, knowledge of the methods of logical 
inquiry and reasoning, and, skill in applying these methods” (p. 5). Glaser concluded his 
discussion by arguing that critical thinking is more than a desirable educational 
objective, it also “helps the individual cooperate with others” (p. 6). In 1961, the 
Educational Policies Commission continued to identify critical thinking as a goal of 
education in The Central Purpose of American Education (Etmis, 1993). 

Sternberg (1986) considered the study of critical thinking to be of particular 
interest because of its joining and melding of three traditions of thought - the 
educational, the psychological, and the philosophical. The educational tradition of 

ERIC 



17 



10 



critical thinking draws heavily upon classroom observation, text analysis, and process 
analysis of thinking in the classroom to guide thinking about critical thinking. Bloom’s 
taxonomy of cognitive skills, dating from 1956, and Gagne’s hierarchy of learning skills, 
dating from 1965, have drawn heavily upon classroom observation (Sternberg, 1986). 

The psychological tradition was concerned with characterizing critical thinking as 
it was performed under the limitations of the person and the environment. In other 
words, the study of critical thinking was valuable in showing how people thought 
critically in the absence of full information, unlimited time, perfect memory, and so 
forth. The psychologists (Bransford, 1984; Bruner, 1960; Feuerstein, 1980; & Sternberg, 
1985) often tested their subjects in laboratory settings, using subjects with select 
limitations in thinking capability (Sternberg, 1986). 

The philosophical tradition focused less upon the requirements of critical 
thinking in the classroom than upon the requirements of formal logical systems. The 
rules of logic can tell us how people might think critically under ideal circumstances in 
which the limitations typically placed upon the human information processing system are 
not in place. However, there are numerous potential limitations that ordinarily block the 
utilization of our full competence - limited time, limited information, limited working 
memory capacity, limited motivation, and so on. Philosophical traditionalists in critical 
thinking were more interested in examining models of competence rather than models of 
performance. Robert Ennis and Richard Paul have been identified as leaders of the 
philosophical group of critical thinking theorists (Sternberg, 1986). 

Ennis (1985) defined critical thinking as “reflective and reasonable thinking that 
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is focused on deciding what to believe or do” (p. 45). He considered critical thinking to 
be a much clearer concept than “higher order thinking skills” (p. 45). Ennis also 
considered higher order thinking skills to be so vague a term that it was useless as a 
guide for the development of teaching, curriculums, and evaluation procedures. It did, 
however, have one persuasive function, according to Ennis - to remind educators that 
there is much more to be learned in school than elementary reading, writing, arithmetic, 
and banks of memorized and soon-to-be-forgotten facts (Ennis, 1985). 

Ennis also considered a further conceptualization of critical thinking as higher- 
order thinking skills. Ennis compared the three top levels of Bloom’s taxonomy 
(analysis, synthesis, and evaluation), and the next two lower levels (comprehension and 
application), as higher order thinking skills. However, he expressed concern over the 
vagueness of the terms and the lack of criteria forjudging outcome of activities to be of 
value in considering the characteristics and abilities of critical thinkers (Ennis, 1985). 

The need for a clear understanding of the critical thinking construct led a 
committee of the American Philosophical Association to undertake a project in 1987 to 
achieve a consensus definition of critical thinking (Facione, 1990). The completed 
“Delphi” research project was published in 1990, representing a consensus definition of 
critical thinking achieved from the anonymous and iterative survey of 46 published 
critical thinking theorists in a variety of disciplines. The following consensus statement 
regarding critical thinking (CT) and the ideal critical thinker is a result of the project: 

We understand critical thinking to be purposeful, self-regulatory judgment which 
results in interpretation, analysis, evaluation, and inference, as well as 
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explanation of the evidential, conceptual, methodological, criteriological, or 
essential as a tool of inquiry. As such, CT is a liberating force in education and a 
powerful resource in one’s personal and civic life. While not synonymous with 
good thinking, CT is a pervasive and self-rectifying human phenomenon. The 
ideal critical thinker is habitually inquisitive, well-informed, trustful of reason, 
open-minded, flexible, fair-minded in evaluation, honest in facing personal 
biases, prudent in making judgements, willing to reconsider, clear about issues, 
orderly in complex matters, diligent in seeking relevant information, reasonable 
in the selection of criteria, focused in inquiry, and persistent in seeking results 
which are as precise as the subject and the circumstances of inquiry permit. 

Thus, educating good critical thinkers means working toward this ideal. It 
combines developing CT skills with nurturing those dispositions which 
consistently yield useful insights and which are the basis of a rational democratic 
society. (APA, 1990, p. 3) 

In 1989, the building momentum to list critical thinking as a core educational 
outcome provoked the National Governors’ Association, then headed by Governor Bill 
Clinton of Arkansas, to incorporate critical thinking among its recommendations for 
national educational goals. In 1990, in response to the Governors’ report, and aware of 
the growing enthusiasm for educational reform. President George Bush, in his state of the 
union address, urged the federal government to adopt National Goals 2000. 

In 1991, the National Center for Education Statistics sponsored a workshop on 
Assessing Higher Order Thinking & Communication Skills in College graduates. The 
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focus of this workshop was to develop a process to develop the goals and the objectives 
for higher education contained in National Education 2000. National Education Goal 5 
directly addressed critical thinking: The goal: “By the year 2000, every adult American 
will be literate and will possess the knowledge and skills necessary to compete in a 
global economy and exercise the rights and responsibilities of citizenship” (Nummedal, 
1991, p. 2), and, the objective: The proportion of college graduates who demonstrate an 
advanced ability to think critically, communicate effectively and solve problems will 
increase substantially” (Nummedal, 1991, p. 2). 

In 1990, NCRIPTAL (the National Center for Research to Improve Postsecondary 
Teaching and Learning) research into over two thousand faculty members’ goals revealed 
a wide range of educational aims. The goals reported by introductory course instructors 
from twelve different college-level fields and disciplines ranged from having the student 
acquire specific knowledge from the academic field to helping students acquire critical 
and independent thinking skills. Instructors whose goals were to help students grow 
intellectually often spoke about broadening the students’ intellectual vistas, teaching 
them to organize and clarify their thinking, helping them develop critical-thinking skills, 
teaching them how to analyze problems and follow logical approaches to situations, and 
encouraging them to discover and express their own thoughts. The task of fostering self 
personal growth in students as expressed in faculty goals, involves helping students 
realize that there is seldom just one answer or one way of solving a problem, that no one 
person can have all the answers, that thinking is fun and challenging, that there are rarely 
simple answers, and that learning is a life-long process (Cameron, 1990). 
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In a research paper discussing critical thinking skills, Jan Corder states: 

Critical thinking skills do not happen automatically. Neither can teachers teach 
students to think critically simply by wanting them to do so. Specific attention 
must be paid to the development of these skills and specific deliberate teaching 
strategies must be employed by the teacher. One cannot coerce a learner to think 
critically or force another to analyze critically the values, beliefs, and 
assumptions on which their lives are built. (Corder, 1992, p. 5) 

With the Clinton administration, plans for a national proficiency examination 
shifted to an individual focus where State legislatures are charged with determining how 
their individual State \vill respond to this mandate. The emphasis has now shifted to two 
specific foci: “How can critical thinking best be taught in our classrooms/clinical 
settings/field settings?” and, “How can we best demonstrate students’ critical fhinking in 
a student assessment project for the purposes of curriculum or program evaluation?” 
(Facione, N., 1995). 

In 1993, the US Department of Education mandated the demonstration of a 
significant improvement in critical thinking in all our nation’s college graduates by the 
year 2000 (National Goals 2000). No specific guidelines for what level of student was to 
be considered a “college graduate,” what constituted “significant improvement,” or how 
that improvement might be demonstrated were identified (Facione, 1995). 

The consensus definition of critical thinking (CT), as defined in the American 
Philosophical Association Delphi report, was confirmed in 1994 by 200 policy-makers, 
employers, and faculty members from two-year and four-year colleges, government 
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agencies, and small and large businesses in a replication study commissioned by the US 
Department of Education Office of Educational Research and Instruction. This 
replication study was carried out from 1993 through 1994 at Pennsylvania State 
University’s National Center on Post-Secondary Teaching, Learning and Assessment. 
The expert consensus definition was strongly endorsed in terms of its descriptions of 
both the skills and disposition of critical thinkers (Jones and Radcliff, 1994). 

The emphasis for continued lifelong learning of critical thinking skills continues. 
On May 28, 1998, the National Education Goals 2000 were updated. The National 
Education Goal 6; Adult Literacy and Lifelong Learning states; “By the year 2000, every 
adult American will be literate and will possess the knowledge and skills necessary to 
compete in a global economy and exercise the right and responsibilities of citizenship” 
(National Education, 7/3/98, p. 1). One of the objectives for National Education Goal 6 
most clearly addressed critical thinking: “The proportion of college graduates who 
demonstrate an advanced ability to think critically, communicate effectively, and solve 
problems will increase substantially” (National Education, 7/3/98, p. 1). 

Assessment of Critical Thinking 

The critical thinking assessment movement can be traced back to Edward 
Glaser’s An Experiment in the Development of Critical Thinking n940V In 1941, 

Glaser and Watson developed the Watson-Glaser Critical Thinking Test (Paul, 1984). 

Paul (1984) stated, “Unlike the case of computer skills or other technical skills, 
there is a natural disinclination for people to recognize the degree to which they 
themselves have not developed critical thinking skill”(p. 6). He goes on to state: 
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Most people, including the most uncritical, take offense at the suggestion that 
they lack skill in this area. This ego-identification with critical thinking (it is the 
others who need it) is a continual problem in the nurturing of such skills. To the 
extent that people lack critical thinking skills, they conceptualize those who have 
them as prejudiced, close-minded, overly academic, negative or nit-picky ... It is 
well to emphasize that the ability to think critically is a matter of degree. No one 
is without any critical skills, (p. 7) 

The Del phi Report. Peter Facione (1990) lists four recommendations which 
address the assessment of critical thinking (CT). 

Recommendation 8: Direct instruction in CT and assessment of CT should be an 
explicit parts of any course granted approval for purposes of satisfying CT 
requirements, whether that course is a CT course per se or a course in a given 
subject field. The primary academic criterion in the evaluation of a proposed 
instructional program for purposes of achieving the CT goal should be whether 
the program will further the development of students’ CT skills and dispositions, 
(p. 16 - 17) Recommendation 11: . . . Minimum CT proficiency expectations 
should be set for each additional education level, including promotion in grade, 
high school graduation, college, entrance, and graduate school admission, (p. 1 8) 
Recommendation 12: In evaluating the acceptability of a CT assessment strategy 
or instrument one should consider content validity, construct validity, reliability, 
and fairness, (p. 19) Recommendation 13: CT assessment should occur 
frequently, and it should be used diagnostically as well as summatively. Different 
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kinds of instruments should be employed, depending on which aspect of CT is 
being targeted and where students are in their learning -- the introductory stage, 
the practice stage, the integration stage, or the generalized transfer stage, (p. 20) 
Paul (1991), at a workshop on Assessing Higher Order Thinking &, 
Communication Skills in College Graduates, reviewed the status of critical thinking 
research. Critical thinking research has emphasized three basic needs for all learning: 
For all students to reason out all basic concepts and understandings, to reason to all 
basic conclusions and solutions, and to reason through and across the curriculum. This 
emphasis has been embedded in the structure of the 1 1 major international conferences 
on research into critical thinking and educational reform (1980 - 1991) held at Sonoma 
State University, attracting 1400 registrants from 20 countries and involving over 350 
sessions representative of a wide variety of academic disciplines. This same emphasis is 
reflected in the 25 or so other conferences focused on critical thinking in the last ten 
years (at Harvard, the University of Chicago, Montclair State, Oakton College, and 
elsewhere) and in most of the articles published concerning critical thinking. He goes on 
to state, “The research into critical thinking has focused not only on the cultivation of 
reasoning in all disciplines but also on generalizable standards for the assessment of 
reasoning as well” (Paul &. Nosich, 1991, p. 2). 

Susan Nummedal (1991), at a workshop on Assessing Higher Order Thinking «& 
Communication Skills in College Graduates, stated: 

We simply must have baseline data for the skills and dispositions we choose to 
assess. We need to know where the students start with respect to these specific 
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skills and dispositions in order to make sense out of the data describing where 
they end up. Ideally, the assessment should have a longitudinal design with the 
same individuals measured over the course of college years. Given the 
difficulties associated vvath such a design, there should be at least be a cohort 
design to provide a picture of the entry level higher order thinking skills and the 
changes that occur over the college years, (p. 14) 

Paul (1991) identified limitations in all twelve of the commercially available 
critical thinking tests as instruments for assessing higher order thinking: 

Cornell Class Reasoning Test, Form X (1964) 

Cornell Conditional Reasoning Test Form X (1964) 

Cornell Critical Thinking Test, Level X (1985) 

Cornell Critical Thinking Test, Level Y (1985) 

The Ennis-Weir Critical Thinking Essay Test (1985) 

Judgement; Deductive Logic and Assumption Recognition (1971) 

Logical Reasoning (1955) 

New Jersey Test of Reasoning Skills (1983) 

Ross Test of Higher Cognitive Processes (1976) 

Test on Appraising Observations (1983) 

Watson-Glaser Critical Thinking- Appraisal (1980) 

None of these tests were designed to serve as a national assessment tool which 
establishes national standards in higher order thinking and motivating and guiding 
instruction (Paul & Nosich, 1991). 
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Paul (1991) goes on to state, “The national assessment proposed would offer a 
range in interdisciplinary items, constructed by experts well versed in a rich and 
substantive concept of critical thinking. The minimal recommendation is that all 
portions of the assessment be given to a representative sample of the student population 
at each educational institution” (p. 26). The assessment proposed by Paul would include 
two broad areas of testing; a machine-gradable portion that includes multiple choice 
items and multiple-rating items and an essay portion. Paul proposed that the test should 
last about three hours, with a total of at least 30 items, two of these being short essay. He 
felt the instrument should be administered by a private agency whose personnel had 
critical thinking credentials increase with reliability (Paul & Nosich, 1991). 

Ennis (1993) stated, “Since the early 1980's, attention to critical thinking 
instruction has increased significantly - with some spillover to critical thinking 
assessment, an area that has been neglected even more than critical thinking instruction” 

(p. 179). He goes on to state “Critical thinking assessment, albeit difficult to do well, is 
possible” (p. 179). Two subthemes are that the difficulties and possibilities vary with the 
purpose of the critical thinking assessment and the format used, and there are numerous 
traps for the unwary. The assessment information would enable institutions to target 
instruction to remediate weaknesses and to build on strengths, as well as to measure what 
students were gaining as a result of attending their classes. 

Ennis (1993) listed seven purposes for critical thinking assessment: 

1 . Diagnosing the levels of students’ critical thinking. 

2. Giving students feedback about their critical thinking prowess. 
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3. Motivating students to be better at critical thinking. 

4. Informing teachers about the success of their efforts to teach students to think 
critically. 

5. Doing research about critical thinking instructional questions and issues. 

6. Providing help in deciding whether a student should enter an educational 
program. 

7. Providing information for holding schools accountable for the critical th ink ing 
prowess of their students, (p. 180 - 181) 

Purposes 6 and 7 are frequently called “high stakes” testing, because so much often 
depends upon the results. Examples of “high stakes”tests, which include sections for 
measurement of critical thinking, are the American College Te.sts (ACTI . Medical 
Colleges Admissions Test (MCAT), College Board Advanced Placement (AP) tests. 
Graduate Record Examination (GRE), and the Law School Aptitude Test (LSAT) (Ennis, 
1993). 

Ennis (1993) goes on to describe the traps educators need to be aware of as they 
pursue the above purposes of critical thinking assessment: 

1. Tests results may be compared with norms, and the claim made that the 
difference, or similarity, is the result of instruction. 

2. A pre-test and a post-test may be given without comparing the class to a 
control group. 

3. The use of the same test for the pretest and the posttest has the problem of 
alerting the students to the test questions. On the other hand, the use of 
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3. Motivating students to be better at critical thinking. 

4. Informing teachers about the success of their efforts to teach students to think 
critically. 

5. Doing research about critical thinking instructional questions and issues. 

6. Providing help in deciding whether a student should enter an educational 
program. 

7. Providing information for holding schools accountable for the critical thinking 
prowess of their students, (p. 180-181) 

Purposes 6 and 7 are frequently called “high stakes” testing, because so much often 
depends upon the results. Examples of “high stakes”tests, which include sections for 
measurement of critical thinking, are the American College Tests (ACT), Medical 
Colleges Admissigns Test (MCAT), College Board Advanced Placement (AP) tests, 
Gradyate Record Examination (GRE), and the Law School Antitude Test (LSAT) (Ennis, 
1993). 

Ennis (1993) goes on to describe the traps educators need to be aware of as they 
pursue the above purposes of critical thinking assessment: 

1. Tests results may be compared with norms, and the claim made that the 
difference, or similarity, is the result of instruction. 

2. A pre-test and a post-test may be given without comparing the class to a 
control group. 

3. The use of the same test for the pretest and the posttest has the problem of 
alerting the students to the test questions. On the other hand, the use of 
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different forms of (allegedly) the same test for critical thinking, is probably 
worse, since different forms are actually different tests. Comparability is 
always suspect, since so much depends on the specific content of the test. 

4. Most critical thinking tests are not comprehensive, especially those that are 
easiest to use. 

5. Multiple-choice tests do not adjust for differences in background beliefs and 
assumptions between test maker and test taker. Different beliefs about the 

situation can sometimes result in justifiably different answers to test questions. 

6. Significant results may be expected in too short a time period. Learning to 
think critically takes a long time. 

7. High-stakes purposes often interfere with the validity of a test. 

8. Scarce resources often lead to compromises that affect test validity, (p. 181) 
Although a number of tests incorporate critical thinking, only a few have critical thinking 
(or some aspect of critical thinking) as their primary concern. Several tests assess more 
than one aspect of critical thinking. These general -content-based tests use content from a 
variety of areas with which tests takers are presumed to be already familiar (Ennis, 

1993). 

In 1994, three instruments were widely recognized as appropriate assessment 
tools for the measurement of critical thinking in adults. These were listed in The 
Supplement to the Eleve nth Mental Measurements Yearbook rConnlev & Impara, 1994). 
These assessment instruments were the Watson-Glaser Critical Thinking Appraisal, the 
Cornell Critical Thinking Tests, and the California Critical Thinking Skills Test. 
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The Watson-Glaser Critical Thinking Appraisal (WGCT) provides scoring in six 
areas: Inference, recognition of assumptions, deduction, interpretation, evaluation of 
arguments, and total critical thinking ability (Conoley & Impara, 1994). The WGCT, 
designed for testing with grades 9 - 12 and adult, was published from 1942 - 1985. The 
following is a review of the Watson-Glaser Critical Thinking Appraisal by Gerald C. 
Helmstadter, Professor of Educational Psychology, Arizona State University, Tempe, 
Arizona: 

The careful appraisal of critical thinking should be an extremely important 
endeavor in our society - one worthy of a great deal of careful research adequately 
backed by financial support. For as our world becomes increasingly complex and 
techmcal, the need for individuals with this capability will surely expand. Indeed, 
the development and identification of this characteristic could become the central 
focus of educational and employee selection. Given the possibility for its 
usefulness, it is quite surprising the authors and publishers have not developed 
this test to its fullest potential in the nearly 40 years since its inception. Nor do 
they seem to have made much of an improvement in the 16 years since the 
previous edition. 

In summary, if the user is in agreement with the authors’ definition of critical 
thinking and the way in which the test items operationalize the construct, and 
further, if the user does not need to make precise statements about the level of 
performance of an individual examinee, this instrument is well worth trying. At 
the same time, this reviewer feel that an imaginative researcher who is willing to 
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put forth the necessary effort could develop an alternative instrument which 
would both stimulate and capture the market for the measurement of this 
important characteristic of human endeavor. (Conoley and Impara, 1994, p. 1693 
- 1694) 

The Cornell Critical Thinking Tests stated purpose is to assess general critical 
thinking ability including “induction, deduction, evaluation, observation, credibility 
(statements made by others), assumption, identification, and meaning” (Conoley & 
Impara, 1994, p. 241). The Cornell Critical Thinking Tests , designed for Advanced and 
gifted high school students and college students and adults, were published from 1961 - 
1985. The following is a review of the Cornell Critical Thinking Tests (CCTT) by Jan N. 
Hughes, Associate Professor of Educational Psychology, Texas A&M University, 

College Station, Texas; 

The content of the Cornell Critical Thinking Tests was selected based on Ennis’s 
conceptualization of critical thinking skills. Level X is intended for use with 
elementary and junior high school students, and Level Z is intended for advanced 
and gifted high school students and adults and for college students. Both tests 
consist of multiple-choice items, and the 50 minutes allocated for test 
administration appears adequate for the 71 items on level Z and the 52 items on 
Level Z. Both levels employ a story format that is likely to maintain the test 
taker’s interest. 

The CCTT has many positive aspects. The developers caution the reader against 
using subscale scores to make individual comparisons. This caveat is important 
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given the small number of items on some scales and the lack of evidence from 
factor analytic studies that the subscales measure distinct skills. The developers 
avoid describing the CCTT as a measure of intelligence or innate reasoning 
abilities and, indeed, emphasize on learned skills is laudable. The manual is 
clearly written at a level of sophistication appropriate for the most likely potential 
test users - educators. The developers exercise appropriate caution in their claims 
for the test’s reliability and validity. They warn that if the test is used as a 
criterion for selecting students into graduate school, it should not be the sole 
criterion. 

Despite these positive aspects, there are several weaknesses that limit the test’s 
usefulness. It is disappointing these shortcomings have not been addressed with 
more vigor in the 14 years since the 1971 edition of the test. 

The norms are described as user norms to emphasize that they are not 
representative of some well-described group. The norms do not permit the user 
to compare a student’s or group’s performance to a normative group. However, 
norms are not as important to the use of the CCTT as a teaching evaluation tool, 
the use of the authors recommend most highly. Because the developers suggest 
the best use of the test is in the evaluation of teaching and curriculum 
development, studies demonstrating the test’s sensitivity to instruction would 
provide important validity data. Only a few studies have investigated the test’s 
sensitivity to instruction. These studies are not well conceptualized and the 
results do not lead to any definitive conclusions as to the test’s construct validity. 
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In summary the evidence that the CCTT measures some ability different from 
existing measures of intelligence and achievement to predict success in endeavors 
presumably calling for critical thinking skills would help to assess the construct 
validity of the CCTT. (Conoley & Impara, 1994, p. 241 - 244) 

The third instrument in wide use, the California Critical Thinking Skills Test, was 
designed to be a standardized assessment instrument targeting core critical thinking 
skills at the post-secondary level” (Coloney & Impara, 1994, p. 37). The California 
Critical Thinking Skills Test is based upon the American Philosophical Association’s 
Delphi consensus conceptualization of critical thinking. The California Critical 
Thinking Skills Test, designed for college students and adults, has publication dates from 
1990 - 1992. The following is a review of the California Critical Thinking skills Test by 
Robert F. McMorris, Professor of Educational Psychology and Statistics, State University 
of New York at Albany, Albany, New York; 

In Roget’s Thesaurus (Chapman, 1992, p. 842), a synonym given for “critical” is 
“crucial.” Certainly critical thinking skills are crucial for individuals and society, 
and developing an appropriate test to assess critical thinking skills is no trivial 
task. The California Critical Thinking Skills Test (CCTST) contains 34 multiple- 
choice items with a 45 minute time limit. The items cover a variety of topics: 

Some items are realistic, providing high face validity, but they potentially 
confound reasoning with content; and some are “nonsense,” content-free items 
for those who prefer a more abstract approach. The items seem reasonably 
interesting. 
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A Delphi panel developed a consensus definition of critical thinking (CT), core 
CT skills with examples, and dispositions crucial to becoming a critical thinker. 
The test contains three subscores based on the panel’s work; Analysis, 

Evaluation, and Inference. The developer, Facione, also apologetically offers two 
other subscores. Deductive Reasoning and Inductive Reasoning, bqsed on 30 of 
the 34 items, to meet a California State University objective. (Given the double 
use of most items, we will refer to subscores rather than subtests.) 

The developer conducted several studies using the CCTST at California State in 
Fullerton involving 1,196 undergraduates, 20 instructors, 5 different courses, and 
three academic departments. These data are bases for validity, reliability, and 
norm information. (Coloney & Impara, 1994, p. 37 - 39) 

William B. Michael, Professor of Education and Psychology, University of Southern 
California, Los Angeles, California, adds the following: 

The CCTST would appear to possess substantial content validity - perhaps more 
than any other competing instrument in light of the collective wisdom of the 
eminent scholars who contributed to its development. The resulting score 
distributions, which are normal in form, provide a basis for differentiating quite 
adequately among the examinees. The 34 items are not easy. On the pretest the 
highest score for the sample of students studied was 29, and on the posttest 3 1 
with a respective means of 15.89 ad 17.27. There may be some concern relative 
to the 45-minute time limit (Conoley & Impara, 1994, p. 39 - 40). 

Peter Facione (1998), author of the California Critical Thinking Skills Test 
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(CCTST), describes the CCTST in the following terms; 

The CCTST presents 34 multiple-choice test items. Each item requires the test 
taker to form a judgment about the best response, from amon^ those provided, to 
a question involving a more or less every day situation or problem. No 
specialized knowledge is required, other than what can reasonably be expected of 
a person who has received a modest high school education AND can read at a 
tenth grade level. No technical vocabulary is used. To consistently answer 
correctly, the person must be skilled at inductive and deductive reasoning and at 
making correct analyses, inferences, and evaluations. The CCTST is a very 
challenging test of CT, but not a test about CT. (Facione, P., 1998, p. 6 - 7) 
Critical Thinking in Nursing 

Almost 40 years ago, nursing leaders developed a strategy to establish a separate 
identity from and professional parity with medicine. They acknowledged the power of 
the methods and values of the dominant culture of medicine by adapting its practices. 

The nursing process emerged from the scientific method, which had long been accepted 
as the only legitimate means of problem solving in medicine. Nursing process became 
the only legitimate means of problem solving in nursing. The nursing process uses 
assessment, diagnosis, intervention, and evaluation. The goal is to systematically and 
objectively apply rules to arrive at the correct solution to the patient’s identified problem. 
Today, virtually every nursing practice textbook is modeled on the nursing process. Most 
nursing curricula identify the nursing process as an essential component, many state 
boards of nursing identify the nursing process as a necessary element in educational 
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program approval, and the National Council Licensure Examination for Registered 
Nurses is based on the nursing process (Jones and Brown, 1993). 

It is critical nurses make effective clinical decisions about patient problems. 
Clinical decision-making or judgment is a highly desirable skill that most educators talk 
about and that many managers complain is nonexistent, but neither is absolutely sure 
how to teach or evaluate. The decision-making process can be logical or illogical, 
deductive or inductive, can be partially taught, and to some degree can be measured 
quantitatively and qualitatively. Certain aspects of nurses’ clinical decision-making can 
be learned and evaluated by examination of decision outcomes and analysis of the 
rationale given for the choice of action or the process by which the decision came about. 
However, numerous factors such as risk, urgency, possible consequences, previous 
experience, available data base, values, and anxiety complicate conclusions drawn from 
any single example of a nurse’s clinical judgment (del Bueno, 1983). 

As members of a profession in which situations change rapidly, nurses cannot 
depend upon routinized behavior, procedure manuals, or traditions to guide clinical 
judgment and decision-making. They must develop the ability to make guided decisions 
drawn from sound, rational bases in order to respond appropriately under the stress of 
fast-paced clinical environments. The development of critical thinking is an 
indispensable component of education for clinical nursing practice. (Malek, 1986). 

In 1989, the National League for Nursing (NLN), the accrediting body for nursing 
education, took a fore-runner position on critical thinking by requiring the demonstration 
of critical thinking in graduates of nursing education programs. “The curriculum 
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emphasizes the development of critical thinking as a specific criterion for accreditation” 
(National League for Nursing, 1989, p. 34). In so doing the NLN positioned nursing to 
lead other professional preparation programs to explore critical thinking criteria for 
learning outcomes, particularly military science, pharmacy, and business (Facione, N., 
1995). 

Miller and Malcolm (1990) expressed the concern of faculty. With the increasing 
emphasis on critical thinking in the nursing curriculum, there is concern that it will 
become another unit of content competing with other units that must be included 
somewhere in the curriculum. Critical thinking is much more than this. It is a 
combination of an attitude of inquiry, supported by a knowledge base and enhanced by 
skill in application. Critical thinking is an approach to inquiry where both students and 
faculty examine clinical and professional issues and search for more effective answers 
(Miller & Malcolm, 1990). 

In 1991, a survey of deans and directors of baccalaureate and higher-degree 
schools of nursing accredited by the National League for Nursing examined perceptions 
of critical thinking as it was then characterized in nursing education. Results indicated 
that critical thinking was conceptualized as a variant of the scientific method. Jones 
(1991) stated, “ . . . the predominant model in baccalaureate education in the United 
States is predicated on critical thinking as a problem-solving activity using principles of 
objectivity, prediction, and control. Critical thinking was usually operationalized as a 
rational-linear process; as a function of deductive logical thinking” (p. 532). Jones went 
on to state: 
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Decision-making in clinical nursing practice, in reality, is more often composed 
of contextually defined value judgments. The problems of everyday nursing 
practice are rarely settled in a rational, linear marmer. More often, nursing 
practice is governed by negotiation between alternative points of view, 
contradictory lines of reasoning, and the realities of situational contingencies 
Nurse educators can no longer provide a sufficient knowledge base of facts to 
circumscribe professional nursing practice. No only are there far too many facts, 
but there are far too many facts which become erroneous over time. (Jones & 
Brown, 1991, p. 533). 

Baker (1992) researched the perceptions of nursing faculty concerning factors 
which promote critical thinking in nursing students. She concluded that nursing faculty 
have a perception of what is required to practice nursing. However, “nurse educators are 
not taught how to teach critical thinking skills. Presently, there are few resources 
available to nursing educators to demonstrate how to develop critical thinking skills in 
the classroom or clinical areas “ (Baker, 1992, p. 150). 

In an editorial in the Journal of Nursing Education Christine Tanner (1993) 

states: 

The new accreditation criteria, if they are implemented in the spirit in which they 
were developed, offer us the opportunity to explore ways that critical thinking 
abilities may be developed and to avoid trivializing important intellectual skills. 

In the press to prepare for an accreditation visit, faculty may be tempted to take 
the pathway we know best - to define critical thinking as nursing process, or to 

er|c 



39 



adopt a standard measure such as the Watson-Glaser Critical Thinking Appraisal 
as the outcome measure. But we will not have learned anything new from our 
own self-study about what we might do differently to help our students develop 
critical thinking, nor will we have engaged ourselves in one of the more 
important aspects of critical thinking - embracing the critical spirit, to have the 
disposition to engage in critical thinking about one’s critical thinking, (p. 99) 
Problem solving, by definition, requires the practitioner to identify the most 
important problem, gather information to explicate the problem, seek a solution to the 
problem, implement a solution, and evaluate the effectiveness of that solution. 

While this model is useful for certain aspects of nursing practice, we would argue 
that the practice of nursing is not so limited in its breadth and depth. Nursing 
decisions are the result of thinking processes, which involve negotiation between 
alternative points of view, contradictory lines of reasoning, and situational 
contexts. Critical thinking is a superordinate concept that includes problem 
solving. In nursing, the need for a knowledge-driven practice base is obvious. 
Nevertheless, it is difficult to determine precisely how much scientific knowledge 
is necessary and sufficient for the delivery of safe, effective patient care. Because 
learning is a life-long process, it is more important that nurses be taught to think 
rather than what to think. (Jones & Brown, 1993, p. 75) 

Carole Brigham (1993) continued on the same theme. 

Nurses must possess a high level of critical thinking skills to practice nursing, 
now and in the future. Nurse educators must teach nurses to be critical thinkers. 
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Nurses must be made aware of the thinking skills they use and continuously strive 
to attain a higher level of these skills. Teaching critical thinking requires a 
reduction of content. A reduction in content requires nurse educators to teach 
concepts and principals, not facts. Metacognition, or the reflecting on one’s own 
thought processes, is necessary in the process of developing critical thinking 
skills. Therefore, nursing curricula must be reformed to overtly include the 
thought processes used by nurses and participatory learning activities if nurse 
educators are to develop a high level of critical thinking skills in nursing students, 
(p. 24) 

In 1993, The Joint Commission for Accreditation of Health care organizations 
established standards that require nurses to participate actively in identifying 
opportunities for improving care and recommending, implementing, and evaluating 
actions to achieve improvement (Case, 1994). This standard describes the nurse 
engaging in critical thinking away from the patient’s bedside (Case, 1994). 

Bette Case (1994) stated: 

The primacy of critical thinking in nursing rests upon decision-making in nursing 
practice. Nursing judgment involves selecting and organizing pieces of data to 
support conclusions. The conclusions we reach lead us to choose and implement 
particular nursing actions from the host of possible nursing actions available to 
us. Although research has not consistently demonstrated a strong relationship 
between critical thinking and clinical judgment, characteristics of critical 
thinking match characteristics of sound clinical decision-making” (p. 101). Case 
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continues, “Critical thinking goes before and beyond problem solving. In critical 
thinking, we emphasize the framing and’reframing of the problem (before 
problem solving). In critical thinking, we also continue to evaluate the solution 
we choose, including reconsidering alternatives not chose initially that might 
become appropriate later as circumstances change (beyond problem solving). 
Problem solving, scientific method, and critical thinking all rely upon generating, 
testing, and assembling evidence to support a variety of solutions, (p. 104) 

Since nursing is faced with facilitating and measuring the critical thinking 
process in direct relationship to nursing, a domain-specific critical thinking definition is 
necessary. However, there is a lack of agreement on the meaning of the concept; it is 
neither clearly understood nor systematically applied. The current definitions originate 
principally from philosophy and education and may not always be relevant to a practice 
discipline such as nursing. The lack of consensus and its relevancy to nursing impedes 
nurse educators who struggle with professional curricula and accreditation expectations 
to define and measure critical thinking in their curricula (Katoaka-Yahira & Saylor, 

1994). 

The conceptual definition of critical thinking, as described in the APA Delphi 
report (1990) contains clear reference to both skills and dispositional attributes 
applicable to nursing knowledge development. Developing a nursing knowledge base by 
carefully examining and delimiting key concepts/constructs and clarifying meanings, 
categorizing phenomena, identifying assumptions, testing 
relationships/hypotheses/theories, as well as conjecturing alternatives for testing, 
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justifying procedures, and stating findings are all manifestations of critical thinking skills 
needed for clinical decision-making in situations that are often high stakes and time 
limited (Facione & Facione, 1996). 

The concern that nursing knowledge should expect a search for best knowledge in 
a given context is a central concern to nursing practice. Nursing practice demands fair- 
mindedness to new evidence and a willingness to reconsider clinical judgments. It 
values a focused and diligent approach to ill-structured patient problems, and requires 
tolerance of multiple perspectives and interpretations when such perspectives and 
interpretations can be supported by reasons and evidence. All these characteristics are 
identified as descriptors of ideal critical thinking disposition (Facione & Facione, 1996). 

“Critical thinking can be characterized as purposeful, self-regulatory judgment, a 
human cognitive process. As such, critical thinking is a pervasive phenomenon that may 
be evident ... in problem solving, professional practice, and everyday life” (Facione & 
Facione, 1996, p. 130). 

Educators should be able to use the language of this consensus definition to 
create teaching aids to engage students and clinicians in critical thinking. In a 
similar manner one should be able to create assessment devices to measure the 
critical thinking component of scientific presentations or demonstrations of 
clinical judgement. (Facione & Facione, 1996, p. 131) 

The buzzword in today’s graduate and undergraduate nursing education is 
“Critical Thinking.” All levels of nursing students are expected to embrace and practice 
this concept in their professional roles. Utilizing critical thinking, the nurse is expected 
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to problem solve, develop ideas, and acquire knowledge in order to educate the public as 
well as other health care professionals (Klein, 1998). 

Professionals are expected to exercise sound, unbiased judgment in interpreting 
and analyzing information, determining the nature of problems, identifying and 
evaluating alternative courses of action, making decisions, and throughout, monitoring 
the process and impact of their problem solving activity so as to amend, revise, correct, 
or alter their decisions, or any element that led up to those decisions, as deemed 
necessary. Judgment in professional practice, correctly exercised, is a reflective, self- 
corrective, purposeful thinking process which requires the professional to take into 
account content knowledge, context, evidence, methods, conceptualizations, and a 
variety of criteria and standards of adequacy (Facione, Facione, & Giancardo, 1998). 

To become a professional nurse requires that you learn to think like a nurse. 

What makes the thinking of a nurse different from a doctor, a dentist, or an engineer? It 
is how nurses view the client and the type of problems nurses deal with in practice when 
they engage in client care. To think like a nurse requires that nurses learn the content of 
nursing; the ideas, concepts and theories of nursing and that they develop intellectual 
capacities and skills so that they, nurses, become disciplined, self-directed, critical 
thinkers (Critical Thinking, 1998). The nursing faculty at Cariboo, British Columbia, go 
on to state: 

Practitioners in nursing who are critical thinkers value and adhere to intellectual 
standards. Critical thinkers strive to be clear, accurate, precise, logical complete, 
significant and fair when they listen, speak, read, and write. Critical thinkers 
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think deeply and broadly ... As nurses, we want to eliminate irrelevant, 
inconsistent, and illogical thoughts as we reason about patient care. Nurses use 
language to clearly communicate in-depth information that is significant to 
nursing care (Critical Thinking, 1998, p. 1). 

Critical Thinking Assessment in Nursing 

Assessment in professional education relies almost exclusively on written and 
oral testing of a predetermined set of cognitive and analytical skills. This is consistent 
with the traditional view that professional activity consists of finding the unique solution 
of a well defined problem by means of these skills. Faculty can point to the fact that a 
good deal of assessment actually exists. It is assessment of the most traditional kind: 
Written courses and comprehensive examinations which test the students’ grasp of basic 
principles and of pertinent facts. Elman and Lynton (1985) state, “ We are beginning to 
recognize that this approach no longer is adequate to the complexity of modem society. 
Professional education and assessment must focus more on the process of defining 
problems and making instrumental judgements, using a variety of real and simulated 
clinical experiences” (p. 2). Elman and Lynton (1985) go on to state that any serious 
examination of assessment in higher education must evaluate the degree programs which 
are intended to prepare students for specific careers. Analysis of the role of assessment 
in undergraduate education must therefore pay a good deal of attention to the career 
oriented programs in our professional schools and departments . 

A committee of the National Academy of Education, The Nation’s Report Card , 
recommended in 1987 that there be a strong effort to develop subject-specific higher 
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order thinking tests (Ennis, 1993). Subject-specific critical thinking tests assess critical 
thinking within one standard subject matter area. A full understanding of any subject 
matter area requires that the person be able to think well in that area. However, as of 
summer, 1993, there were no subject-specific critical thinking tests (Ennis, 1993). 

Janet Hickman (1993) conducted an assessment of critical thinking assessment in . 
nursing education. She reviewed 24 studies of critical thinking in nursing education 
conducted between 1977 and 1990. Six studies presenting longitudinal data provided 
mixed findings regarding the impact of nursing education on critical thinking. Four of 
these studies found no significant gains in critical thinking over time. Nine studies 
presented cross-sectional data about nursing education and critical thinking ability. Six 
of those studies support the impact of nursing education on critical thinking. The other 
three studies demonstrated no significant differences between groups studied. Ten 
studies of critical thinking and clinical judgement presented mixed and contradictory 
results, varying from a weak positive relationship to no relationship. One study 
demonstrated a negative relationship between empathy and critical thinking ability. 
Hickman went on to state, “Research to identify the impact of nursing curricula on the 
development of critical thinking in students is still embryonic, as is the relationship 
between critical thinking and the formation of clinical judgements” (p. 46). 

It is significant to note the lack of support for curricular impact on critical 
thinking: 

The Watson-Glaser CTA was developed to provide a sample of the ability to 
think critically about statements encountered in daily work, magazines, and 
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newspapers. Consequently, it is not surprising that it would be difficult to show 
how a specific curriculum, course or teaching strategy affects such general 
critical abilities in a different context than what was studied in class (Watson, 
1964, p. 10). 

Hickman states, “This comment makes a strong and important point, especially in view 
of the fact that the majority of nursing research relies on the Watson-Glaser Critical 
Thinking Appraisal scores. It certainly provides a rationale for the mixed research 
findings related to nursing education and its impact on critical thinking” (p. 42). In 
summary, Hickman concluded; 

There is not a strong research base supporting a relationship between nursing 
curricula and critical thinking. It may be that this is due to lack of an appropriate 
instrument to measure critical thinking in nursing. Research related to skill and 
intellectual development suggests that critical thinking may be an unrealistic 
expectation ... It is clear that research efforts should be directed at developing an 
appropriate instrument to measure critical thinking that occurs in nursing 
practice, (p. 46) 

It is also significant to note, in Hickman’s research summary of 24 studies of 
critical thinking in nursing education, only twelve studies included associate degree 
nursing students or graduates of associate degree nursing programs. None of the study 
samples were limited to only associate degree nursing students; of the twelve studies, all 
compared associate degree samples with diploma and/or baccalaureate samples. Three 
of the studies presented longitudinal data: Gross et al found no significant gain; Kentgen- 
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Andrews found no significant gain; and, Poole found an increase only in the low 
cognitive domain (Hickman, 1993). The other nine studies compared critical thinking 
and/or clinical judgment among various combinations of nursing program students. 
Baccalaureate nursing students seemed to consistently score higher on the Watson-Glaser 
Critical Thinking Appraisal tests. However, the results were mixed when comparing 
critical thinking and clinical judgment of associate degree nursing students with other 
nursing programs (diploma or baccalaureate) (Hickman, 1993). 

In 1997, the National League for Nursing Accreditation published the Interpretive 
Guidelines for Standards and Criteria: 1997 Associate Degree Programs in Nursing. In 
these guidelines. Criterion #14 addresses educational effectiveness. Criterion 14 states: 
“The required outcomes and optional outcomes, as they relate to the program type, are 
evaluated” (NLN, 1998, p 2). The first of four “Required Outcomes” listed is critical 
thinking. The National League of Nursing’s Interpretive Guidelines for Standards and 
Criteria: 1997 Associate Degree Programs in Nursing defines critical thinking as “How 
faculty define students’ skills in reasoning, analysis, and decision-making relevant to the 
discipline of nursing” (NLN Glossary, 1998, p. 2). 

“Outcomes,” as defined by the NLN (1998), means “the evidence showing the 
degree to which course/or level purposes and objectives are or are not being attained and 
includes appropriate skills and competencies by students at the completion of their 
educational program” (p. 4 - 5). Examples of procedures for assessing an instructional 
program include the following, “Pre- and post-testing of students ... as indicators of 
outcomes of critical thinking such as, California Critical Thinking Dispositions Inventory 
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and California Critical Thinking Skills Tests” (NLN, 1998, p. 4). 

Noreen Facione (1997) reported the results of an analysis of a longitudinal 
aggregate data set comprising 145 predominately undergraduate samples contributed by 
on-site collaborative investigators at fifty programs of nursing education throughout the 
United States. This sample did not include associate degree nursing programs. The data 
analyzed were collected from 1992 through 1997. Significant relationships are reported 
between the California Critical Thinking Skills Test (CCTST) and a wide variety of 
academic achievement indicators (e.g. grade point average, standardized test scores), 
student descriptors (e.g. age, sex, RN-status, NCLEX [National Nursing Licensure 
Examination] passage), and program descriptors), e.g. student to faculty ratio, location of 
program, faculty focus on critical thinking in planning and curriculum development. 
Modest cross sectional increases and longitudinal gains are demonstrated in critical 
thinking skills. Initial percentile norms for the CCTST were calculated for nursing 
students by undergraduate class level (Facione, 1997). 

Critical Thinking at Madison Area Technical Colleg e 

Madison Area Technical College (MATC), along with other vocational technical 
colleges, is considered to be a “higher education institution” by North Central 
Association (NCA). NCA refined its understanding of general education and its 
relationship to the total curriculum and approved this statement of General Education: 
General Education is “general” in several clearly identifiable ways: It is not 
directly related to a student’s formal technical, vocational, or professional 
preparation; it is part of every student’s course of study, regardless of his or her 
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area of emphasis, and it is intended to impart common knowledge, intellectual 
concepts, and attitudes that every educated person should possess. (Core 
Abilities and You, 1996, p. I) 

The Core Abilities ... are intended to meet the NCA requirements and to better 
articulate MATC’s commitment to equipping students to become productive, civic 
minded citizens. The original eight core abilities were developed by a state-wide 
General Education Task Force composed of representatives from all sixteen districts. 

They were further refined by the MATC Philosophy of General Education Committee, 
composed of representatives from all divisions of the College. The MATC Core 
Abilities, Liberal Studies, and Student Learning Outcomes Steering Committees have 
elaborated on these eight core abilities. General Education provides comprehensive 
instruction in all these areas; however, courses in occupational programs also address 
these abilities (Core Abilities and You, 1996). 

The eight Core Abilities are: Communication; Critical Thinking; Ethics; Global 
Awareness; Mathematics; Science and Technology; Self-Awareness; and Social 
Interaction. Critical Thinking is further defined: Students demonstrate critical 
thinking when they use multiple perspectives to: 

1 . Demonstrate observation skills 

2. Identify a problem to be solved, task to be performed, or decision to be made. 

3. Exhibit personal, professional, and academic honesty. 

4. Recognize their responsibility to personal, social, professional, educational, 
and natural environments and make informed decisions based on that 
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responsibility. 

5. Display behavior consistent with the ethical standards within a discipline or 

profession. (Core Abilities and You, 1996, p. 2) 

Critical Thinking in Madison Area Technical College Associate Degree Nursing 
Program 

The MATC Associate Degree Nursing Advisory committee, at the April, 1995 
meeting, identified critical thinking skills as essential competencies for the Associate 
Degree Nursing graduate. In May, 1995, critical thinking skills were verified as essential 
competencies in the Associate Degree Nurse DACUM for Community Health Nurses 
(Develop A CurriculM, 1995). DACUM’s are developed through a Continuous 
Improvement process to identify first competencies and then components to accomplish 
these competencies. The Community Health Nurses’ DACUM was developed by staff 
nurses currently working in community health, and, also, who were Associate Degree 
Nursing graduates. 

On March 1, 1996, at an Associate Degree Nursing faculty retreat, critical 
thinking was identified as a major content thread to be carried as a priority through all 
four semesters of the Associate Degree Nursing program of instruction for Madison Area 
Technical College. The Program Evaluation committee continues to discuss how best to 
assess critical thinking in the program. 

Summary 

A review of the literature has shown the development of critical thinking as a 
well defined set of skills to be utilized not only in education but in life long learning, as a 
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member of a profession, and as a contributing citizen. Critical thinking is included in the 
national goals for education. The need for a nationalized standard for the assessment of 
critical thinking was made apparent by Nummedahl, Paul, and Facione. However, there 
are no subject-specific critical thinking tests. Within the profession of nursing, the 
assessment of critical thinking was made mandatory by the National League of Nursing, 
in 1989. Madison Area Technical College General Education division and the Associate 
Degree Nursing program both support core competencies in critical thinking. 

Conclusions were reached that there is a strong relationship between nursing 
curriculum and critical thinking. Over 24 studies of assessment of critical thinking in 
nursing education were reviewed. Of these studies, only 12 included associate degree 
nursing students in their sample population, for the purpose of comparing the 
performance of associate degree nursing students with other levels of nursing students. 
Only three longitudinal studies were conducted with associate nursing students, with 
mixed results. 

In 1997, the California Critical Thinking Skills Test was identified by the 
National League of Nursing as an appropriate instrument for assessment of critical 
thinking skills Facione reported upon a longitudinal aggregate data analysis of 
assessment of critical thinking skills in nursing students, using the California Critical 
Thinking Skills Test. The purpose of this study was to develop a standard for critical 
thinking skills for nursing students and nursing programs. However, associate degree 
nursing students were not included in this study. Other than these studies, assessment of 
critical thinking skills of associate degree nursing students is lacking. Assistance is 
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needed for associate degree nursing programs to set standards related to critical thinking 
outcomes for their educational programs. Formal studies are needed. 
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Chapter III 



Methodology 



Introduction 

This chapter will describe the research hypotheses, the subjects under study, and 
how they were selected for inclusion in this study. In addition, the instruments being 
used to collect information will be discussed as to their content, validity, and reliability. 
Data collection and analysis procedures will then be described. The chapter will 
conclude with a discussion of methodology. 

Research Hypotheses 

The following hypotheses are proposed: 

1 . There will be an increase in the critical thinking skills of Associate 
Degree Nursing students from entry to completion of the Associate 
Degree Nursing program at Madison Area Technical College - Reedsburg. 

2. There will be an increase in the critical thinking skill subset “Analysis” in 
Associate Degree Nursing students from entry to completion of the 
Associate Degree Nursing program at Madison Area Technical College - 
Reedsburg. 

3. There will be an increase in the critical thinking skill subset “Evaluation” 
in Associate Degree Nursing students from entry to completion of the 
Associate Degree Nursing program at Madison Area Technical College- 
Reedsburg. 
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4. There will be an increase in the critical thinking skill subset “Inference” 
in Associate Degree Nursing students from entry to completion of the 
Associate Degree Nursing program at Madison Area Technical College - 
Reedsburg. 

5. There will be an increase in the critical thinking skill subset “Deductive 
Reasoning” in Associate Degree Nursing students from entry to 
completion of the Associate Degree Nursing program at Madison Area 
Technical College- Reedsburg. 

6. There will be an increase in the critical thinking skill subset “Inductive 
Reasoning” in Associate Degree Nursing students from entry to 
completion of the Associate Degree Nursing program at Madison Area 
Technical College- Reedsburg. 

Research Design 

The study design is a combination of quasi-experimental and descriptive research. 
The study used a nonequivalent control group design format, as there was no random 
assignment to groups. The two groups were already in existence. The two existing 
groups were pre-tested; the MATC Associate Degree Nursing program was the treatment, 
the same two groups were post-tested. However, the researcher did not manipulate the 
independent variables (the nursing program). Thus, the study is descriptive research - 
describing what happens as a result of participating in the MATC Associate Degree 
Nursing program. 

The MATC Associate Degree Nursing Advisory committee, at the April, 1995 
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meeting, identified critical thinking skills as essential competencies for the Associate 
Degree Nursing graduate. At this time, several faculty had participated in a Critical 
Literacy seminar as an inservice activity at MATC. However, critical thinking was not 
part of the program curriculum. Faculty did express the belief they were teaching 
students to think critically, although no formal instruction in critical thinking was 
included in the program plan. 

On March 1, 1996, at an Associate Degree Nursing faculty retreat, ADN faculty 
spent an entire day learning about critical thinking, the application to nursing, and means 
for implementing instructional strategies related to critical thinking in nursing education. 
The faculty did not reach consensus regarding a definition for critical thinking. However, 
critical thinking was identified as a major content thread to be carried as a priority 
through all four semesters of the Associate Degree Nursing program of instruction for 
Madison Area Technical College. The actual implementation of this curricular plan did 
not take place until August, 1997. None of the participants in this current study 
participated in the revised curriculum, although faculty did increase their own perception 
and understanding of critical thinking. Some faculty were piloting critical thinking 
instructional exercises and were increasing their efforts to improve critical thinking in 
clinical situations. 

Sample Description and Selection 

A cohort of twenty-four students entering the Associate Degree Nursing program 
in August, 1995 and a second cohort of twenty four students entering in August, 1996, at 
the Madison Area Technical College - Reedsburg campus were selected as the sample for 
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Madison Area Technical College’s Associate Degree Nursing program. Each entering 
class was selected through a randomized selection process from the first day applicants 
who fulfilled all the prerequisites for entry. The program has open enrollment to all 
applicants who meet the prerequisites. The prerequisites are: A minimum grade of C in a 
college level chemistry course and an algebra course, completion of the ACT test 
(however, no minimum score is required), and current certification in CPR (Cardio- 
pulmonary Resuscitation). No qualified applicant was refused the right to participate in 
the randomized selection process conducted by the computer support services at Madison 
Area Technical College - Truax. However, the entering class size is limited to twenty- 
four students per year at the Reedsburg campus. The acceptance process for all program 
students is the same regardless of the campus to which they apply. No student who 
meets the program prerequisites is refused admittance. However, the student may be on 
a waiting list for one or two years, prior to entering the program. 

Instrumentation 

Demographic data, e.g. age, grade level of education upon entering program, 
highest level of education upon entering the program, high school graduation or GED, 
ACT score, and grade point average, was collected through the review of individual 
electronic student records by the researcher. Assessment data was collected through the 
use of the California Critical Thinking Skills Test Form A . The California Critical 
Thinking Skills Test was designed to be “a standardized assessment instrument targeting 
core critical thinking skills at the post-secondary level” (Coloney & Impara, 1994, p. 37). 
The California Critical Thinking Skills Test is based upon the American Philosophical 
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Critical Thinking Skills Test, designed for college students and adults, has publication 
dates from 1990 - 1992. 
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According to the CCTST Fact Sheet (Facione, 1 990), the CCTST measures the 
growth in CT skills which is an intended outcome of completing a college level 
general education CT course. Facione has collected considerable pre and post 
data, some quite cleverly by measuring post at the end of the first semester and 
pre with similar students at the beginning of the second semester. Gain runs 
between .04 and 1.45 in mean scores on a 34-item test, statistically significant 
with large samples. The total score has moderate to good reliability. (Conoley & 
Impara, 1994, p. 38 - 39) 

William B. Michael, Professor of Education and Psychology, University of 
Southern California, Los Angeles, California, adds the following: 

The CCTST would appear to possess substantial content validity - perhaps more 
than any other competing instrument in light of the collective wisdom of the 
eminent scholars who contributed to its development. The resulting score 
distributions, which are normal in form, provide a basis for differentiating quite 
adequately among the examinees. The 34 items are not easy. On the pretest the 
highest score for the sample of students studied was 29, and on the post-test 3 1 
with a respective means of 15.89 ad 17.27. There may be some concern relative 
to the 45-minute time limit. (Conoley & Impara, 1994, pp. 39 - 40) 

Robert F. McMorris, Professor of Educational Psychology and Statistics, State University 
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of New York at Albany, Albany, New York, also addressed content validity in his review 
of the CCTST: “Test users are urged to study the items to judge validity, especially to 
estimate whether the test meets their conceptualizations of critical thinking” (Conoley & 
Impara, 1994, p. 38). 

In 1995, the CCTST was selected as the instrument for this study because it had 
substantial content validity. The 45-minute time limit also made it possible to administer 
the instrument within the 50-minute confines of the MATC classroom structure. In 1998, 
the National League of Nursing identified pre- and post-testing of nursing students using 
the CCTST as an indicator of outcomes of critical thinking in self-studies for 
accreditation of nursing programs. 

Data Collection 

The California Critical Thinking Skills Test Form A was administered to the 
entering Associate Degree Nursing class in August, 1995 and August, 1996, prior to the 
beginning of program instruction. The California Critical Thinking Skills Test Form A 
was also administered at the completion of the program. May, 1997 and May, 1998. The 
test was administered after the last day of classes but prior to final exams. The test was 
administered by the researcher at the Madison Area Technical College - Reedsburg 
campus. The procedures for implementation in The California Critical Thinking Skills 
Test Form A test manual were followed. 

Data Analysis 

The collected data were entered into a spreadsheet, using Excel software. The 
spreadsheet was designed to analyze the data according to frequencies and ranges for 
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each item. For determining pre- and post-test differences, nominal statistical analysis 
was conducted through the use of Minitab for a t-test of differences. 

Limitations 

Students transferring to this program from either another Madison Area Technical 
College campus or from another nursing program were not included in the sample. 
Students who were admitted at a level other than entry level of the program or with 
advanced standing were not included in the sample. 

The sample was limited to those students who remained in the program and who 
graduated from the program. Not all students who entered the program were able to 
complete the program. Students who transferred to another Madison Area Technical 
College campus or Associate Degree Nursing program were not included in the study. 
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Chapter Four 



Results 

Introduction 

This chapter will present the results of assessment of the degree of change in 
critical thinking skills of Associate Degree Nursing students at Madison Area Technical 
College. The demographic information and descriptive statistics will be reported first. 
Data collected on each of the research hypotheses will then be given. 

Demographic Data 

Sample A of this study consisted of twenty Associate Degree Nursing students 
from the Madison Area Technical College - Reedsburg campus. Twenty-four students 
entered the program in August, 1995. Twenty-one students (87.5%) graduated from the 
program in May, 1997. One student did not complete the post-test. No transfer students 
from other Associate Degree Nursing programs or other Madison Area Technical College 
campuses or advance placement students were included in this study. Twenty-four 
students (100%) completed the pretest; twenty students (83.3%) completed the post-test. 
(See Table 1 ; Sample Population). 

Sample B of this study consisted of twenty-three Associate Degree Nursing 
students from the Madison Area Technical College - Reedsburg campus. Twenty-four 
students entered the program in August, 1995. Twenty-three students (95.8 %) graduated 
from the program in May, 1997. No transfer students from other Associate Degree 
Nursing programs or other Madison Area Technical College campuses or advance 




61 



53 

placement students were included in this study. Twenty-three students (95.8%) 
completed the pretest; twenty-three students (95.8%) completed the post-test. (See 
Table 1; Sample Population). 

Table 1 

Sample Population 



Group 


Number 

Entering 


Number 

Graduating 


Number Taking 
Pre-test 


Number Taking 
Post-test 


Sample A 


24 


21 


24 


20 


Sample B 


24 


23 


23 


23 



Sample A consisted of nineteen females (95.0 %) and one male (5.0 %). The 

\ 

\ 

average age of entering students in Sample A was 35.95 years with a range in age from 
23 to 50. (See Table 2: Sample Age). Sample B consisted of twenty two females 
(95.65 %) and one male (4.34 %). The average age of entering students in sample B was 
31.17 with a range in age from 2 1 to 47. (See Table 2: Sample Age). 

Table 2 
Sample Age 







Sex 




Minimum 

Age 




Group 


N 


M 


F 


Mean Age 


Maximum 

Age 


Sample A 


20 


1 


19 


35.95 


23 


50 


Sample B 


23 


1 


22 


31.17 


21 


47 



Student registration requires students to report the highest level of education 
completed by grade level. For Sample A, the highest level of education by grade level 




62 



54 



reported by students upon program entry was grade 16; the lowest level of education by 
grade level reported by students upon program entry was grade 12. The mean level of 
education by grade level upon entry was 12.850. (See Table 3; Highest Level of 
Education by Grade Level). 

For Sample B, the highest level of education by grade level reported by students 
upon program entry was grade 16; the lowest level of education by grade level reported 
by students upon program entry was grade 12. The mean level of education by grade 
level reported by students upon program entry was 13.391. (See Table 3: Highest Level 
of Education by Grade Level). 

Table 3 

Highest Level of Education by Grade T.evel 











Grade Level 




Group 


N 


Mean 


Median 


Minimum 


Maximum 


Sample A 


21 


12.850 


12.500 


12 


16 


Sample B 


23 


13.391 


13.000 


12 


16 



Students were required to submit ACT test scores prior to application for 



admission to the Associate Degree Nursing program. The ACT scores for Sample A 
ranged from a minimum of 14 to a maximum of 25. Sample A students had a mean 
score of 20.35 for the ACT. The ACT scores for Sample B ranged from a minimum of 
14 to a maximum of 29. Sample B students had a mean score of 20. 13 for the ACT. 

Final cumulative grade point averages were collected for Sample A and Sample 
B. The mean grade point average for Sample A was 3. 1465, with a range from 2.61 to 
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3.95; the mean grade point average for Sample B was 2.8961, with a range from 2.36 to 
3.86. (See Table 5: Cumulative Grade Point Average). 

Table 4 
ACT Scores 

ACT Results 



Standard 



Group 


N 


Mean 


Median 


Deviation 


Minimum 


Maximum 


Sample A 


21 


20.35 


21.00 


2.996 


14 


25 


Sample B 


23 


20.13 


20.00 


3.209 


14 


29 



Table 5 

Cumulative Grade Point Average 



Cumulative Grade Point Average 



Standard 



Group 


N 


Mean 


Median 


Deviation 


Minimum 


Maximum 


Sample A 


20 


3.1465 


3.0550 


0.3999 


2.61 


3.95 


Sample B 


23 


2.8961 


2.8900 


0.3549 


2.36 


3.86 



Comparisons of Samples A and B 

Sample A as a group was compared to Sample B as a group using a two-sample t- 
test. The mean age of Sample A was compared to the mean age of Sample B using a two 
sample t-test. Sample A students had a mean age of 35.95; Sample B students had a 
mean age of 3 1 . 1 7. The t value was 1 . 89. Sample A students were slightly older than 
Sample B students; however, this difference was not statistically significant. (See Table 
6: Comparison of Samples A and B). 
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The mean level of education by grade level reported by students in Sample A was 
compared to the mean level of education by grade level reported by students in Sample B 
using a two sample t-test. Sample A students had a mean grade level of 12.85 and 
Sample B students had a mean grade level reported of 13.39. Highest level of education 
by grade level for Sample B was slightly higher than Sample A. The t-value was -1.36; 
there was no statistical significance. (See Table 6; Comparison of Samples A and B). 

The mean ACT score of Sample A was compared to the mean age of Sample B 
using a two sample t-test. Sample A ACT scores had a mean of 20.35 and Sample B 
ACT scores had a mean of 20. 13. ACT scores were almost identical for both groups and 
there was no significant difference. (See Table 6: Comparison of Samples A and B). 
Table 6 



Comparison of Samples A and R 





Sample A 


Sample B 






Comparison 


N 


Mean 


N 


Mean 


t-test 


Probability 


Age 


20 


35.95 


23 


31.17 


1.89 


0.065 


Highest Level of Education 


20 


12.85 


23 


13.39 


-1.36 


0.28 


ACT Score 


20 


20.35 


23 


20.13 


0.23 


0.82 


Grade Point Average 


20 


3.146 


23 


2.896 


2.16 


0.037 



The mean grade point average of Sample A was compared to the mean grade 
point average of Sample B using a two sample t-test. Sample A students had a mean 
grade point average of 3. 146 and Sample B students had a mean grade point average of 
2.896. The grade point average for Sample A was significantly higher (0.25 higher) than 
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Sample B s grade point average. (See Table 6: Comparison of Samples A and B). 
Anecdotally, faculty reported Sample A to be a more motivated group than Group B. 
Group B, as a whole, was perceived to have poorer study skills, less motivation to study, 
worked too much, and tested poorly. 

California Critical Thinking Skills Test Data 

Possible scores on the instrument, the California Critical Thinking Skills Test 
(CCTST), are a Total score from 1 to 34. Sub-scale scores on the instrument can range as 
follows. Analysis (1 to 9), Evaluation (1 to 14); Inference (1 to 11); Deductive Reasoning 
(1 to 16); and Inductive Reasoning (1 to 14). The sum of the three Delphi construct 
scores - analysis, evaluation, and inference - is equal to the total score (Facione and 
Facione, 1992). 

Research Results 
Pre-test Scores 

The instrument, the California Critical Thinking Skills Test (CCTST), was 
administered as a pre-test, prior to classroom instruction in the Associate Degree Nursing 
program. The CCTST pre-test mean Total score for Sample A was 14.95. Their Total 
test scores ranged from 1 1 as a minimum to 21 as a maximum. Sample A’s subset 
Analysis had a mean score of 4.250. Their Analysis scores ranged from 2 as a minimum 
to 7 as a maximum. Sample A’s subset Evaluation had a mean score of 5.00. Their 
Evaluation test scores ranged from 1 to 9. The subset Inference had a mean score of 
5.850. The Inference test scores ranged from 3 to 9. Sample A’s subset Deductive 
Reasoning had a mean score of 7.050, with test scores ranging from 3 to 12. The subset 
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Inductive Reasoning had a mean score of 6.450 and test scores ranging from 2 to 11. 
(See Table 7: Sample A CCTST Pre-test Scores). 

Table 7 

Sample A CCTST Pre-test Scores 



Sample A Pre-test Scores 



Test Area 


N 


Mean 


Median 


Standard 

Deviation 


Minimum 


Maximum 


Total 


20 


14.95 


14 


3.252 


11.000 


21.000 


Analysis 


20 


4.250 


4 


1.333 


2.000 


7.000 


Evaluation 


20 


5.000 


5.000 


2.176 


1.000 


9.000 


Inference 


20 


5.850 


6.000 


1.496 


3.000 


9.000 


Deductive 

Reasoning 


20 


7.050 


6.000 


2.523 


3.000 


12.000 


Inductive 

Reasoning 


20 


6.450 


6.500 


2.188 


2.000 


11.000 



The CCTST pre-test mean Total score for Sample B was 14.957. Their Total test 
scores ranged from 10 as a minimum to 20 as a maximum. Sample B’s subset Analysis 
had a mean score of 4.348. Their Analysis scores ranged from 2 as a minimum to 6 as a 
maximum. Sample B’s subset Evaluation had a mean score of 5.043. Their Evaluation 
test scores ranged from 2 to 9. The subset Inference had a mean score of 5.565. The 
Inference test scores ranged from 2 to 8. Sample B’s subset Deductive Reasoning had a 
mean score of 7.565, with test scores ranging from 3 to 14. The subset Inductive 
Reasoning had a mean score of 5.870 and test scores ranging from 2 to 9. (See Table 8: 
Sample B CCTST Pre-test Scores). 
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Table 8 

Sample B CCTST Pre-test Scores 



Sample B Pre-test Scores 



Test Area 


N 


Mean 


Median 


Standard 

Deviation 


Minimum 


Maximum 


Total 


23 


14.957 


15.000 


2.992 


10.000 


20.000 


Analysis 


23 


4.348 


4.000 


1.071 


2.000 


6.000 


Evaluation 


23 


5.043 


5.000 


1.846 


2.000 


9.000 


Inference 


23 


5.565 


6.000 


1.502 


2.000 


8.000 


Deductive 

Reasoning 


23 


7.565 


7.000 


2.793 


3.000 


14.000 


Inductive 

Reasoning 


23 


5.870 


6.000 


1.632 


2.000 


9.000 



These results for pre-tests for both Sample A and Sample B are similar to those of 
a sample of freshman students entering BSN programs (Facione, 1997, p. 18). The 
average test score for BSN students was 14.5 as compared to 14.95 for Sample A and 
14.957 for Sample B. This difference is not statistically significant. MATC has a more 
homogeneous student group than BSN students. Ranges for MATC students were 10 
points, while freshman BSN students were 20 points. MATC’s students had higher 
minimum total scores (1 1 and 10 for MATC versus 5 for BSN) and lower maximum 
scores than freshman BSN students (21 and 20 for MATC versus 25 BSN). 

Post-test Scores 

The instrument, the California Critical Thinking Skills Test (CCTST) was 
administered as a post-test prior to final examinations in the fourth and final semester of 
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the Associate Degree Nursing program. The CCTST post-test mean Total score for 
Sample A was 15.85. Their total test scores ranged from 1 1 as a minimum to 2 1 as a 
maximum. Sample A’s subset Analysis had a mean score of 4. 100. Their Analysis 
scores ranged from 1 as a minimum to 7 as a maximum. Sample A’s subset Evaluation 
had a mean score of 6.450 Their Evaluation test scores ranged from 1 to 10. The subset 
Inference had a mean score of 5.300. The Inference test scores ranged from 0.00 to 9. 
Sample A’s subset Deductive Reasoning had a mean score of 7.950, with test scores 
ranging from 3 to 14. The subset Inductive Reasoning had a mean score of 6.450 and test 

scores ranging from 1 to 10. (See Table 9; Sample A CCTST Post-test Scores). 

Table 9 

Sample A C CTST Po.sNtRst <;r.nrpc 



Sample A Post-test Scores 



Test Areas 


N 


Mean 


Median 


Standard 

Deviation 


Minimum 


Maximum 


Total 


20 


15.85 


15.50 


5.42 


11.000 


21.000 


Analysis 


20 


4.100 


4.000 


1.651 


1.000 


7.000 


Evaluation 


20 


6.450 


6.500 


2.724 


1.000 


10.000 


Inference 


20 


5.300 


5.000 


2.250 


0.000 


9.000 


Deductive 

Reasoning 


20 


7.950 


8.000 


3.486 


3.000 


14.000 


Inductive 

Reasoning 


20 


6.450 


7.000 


2.212 


1.000 


10.000 



The CCTST pos-test mean Total score for Sample B was 16.261. Their total test 



scores ranged from 10 as a minimum to 24 as a maximum. Sample B’s subset Analysis 
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the Associate Degree Nursing program. The CCTST post-test mean Total score for 
Sample A was 15.85. Their total test scores ranged from 1 1 as a minimum to 2 1 as a 
maximum. Sample A’s subset Analysis had a mean score of 4. 100. Their Analysis 
scores ranged from 1 as a minimum to 7 as a maximum. Sample A’s subset Evaluation 
had a mean score of 6.450 Their Evaluation test scores ranged from 1 to 10. The subset 
Inference had a mean score of 5.300. The Inference test scores ranged from 0.00 to 9. 
Sample A’s subset Deductive Reasoning had a mean score of 7.950, with test scores 
ranging from 3 to 14. The subset Inductive Reasoning had a mean score of 6.450 and test 
scores ranging from 1 to 10. (See Table 9: Sample A CCTST Post-test Scores). 

Table 9 

Sample A CCTST Post-test Scores 

Sample A Post-test Scores 



Standard 



Test Areas 


N 


Mean 


Median 


Deviation 


Minimum 


Maximum 


Total 


20 


15.85 


15.50 


5.42 


11.000 


21.000 


Analysis 


20 


4.100 


4.000 


1.651 


1.000 


7.000 


Evaluation 


20 


6.450 


6.500 


2.724 


1.000 


10.000 


Inference 


20 


5.300 


5.000 


2.250 


0.000 


9.000 


Deductive 

Reasoning 


20 


7.950 


8.000 


3.486 


3.000 


14.000 


Inductive 

Reasoning 


20 


6.450 


7.000 


2.212 


1.000 


10.000 



The CCTST post-test mean Total score for Sample B was 16.261. Their total test 
scores ranged from 10 as a minimum to 24 as a maximum. Sample B’s subset Analysis 
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had a mean score of 4.739. Their Analysis scores ranged from 2 as a minimum to 8 as a 
maximum. Sample B’s subset Evaluation had a mean score of 5.696. Their Evaluation 
test scores ranged from 2 to 11. The subset Inference had a mean score of 5.783. The 
Inference test scores ranged from 3 to 9. Sample A’s subset Deductive Reasoning had a 
mean score of 7.652, with test scores ranging from 3 to 13. The subset Inductive 
Reasoning had a mean score of 6.696 and test scores ranging from 3 to II. (See Table 
10: Sample B CCTST Post-test Scores). 

Table 10 

Sample B CCTST Post-test Scores 

Sample B Post-test Scores 



Standard 



Test Area 


N 


Mean 


Median 


Deviation 


Minimum 


Maximum 


Total 


23 


16.261 


16.000 


4.741 


10.000 


24.000 


Analysis 


23 


4.739 


5.000 


1.42 1 


2.000 


8.000 


Evaluation 


23 


5.696 


4.000 


2.601 


2.000 


11.000 


Inference 


23 


5.783 


6.000 


1.858 


3.000 


9.000 


Deductive 

Reasoning 


23 


7.652 


8.000 


2.994 


3.000 


13.000 


Inductive 

Reasoning 


23 


6.696 


7.000 


2.098 


3.000 


11.000 



Total CCTST mean scores were reported by Facione (1997, p. 19-21) for 
sophomore (15.7), junior (16.1 1), and senior (16.4) BSN students. These compare to 
mean total scores of 15.85 and 16.261 for the graduating MATC ADN students. Facione 
suggests the CCTST scores should increase as students progress through nursing 
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programs. However, she adds; “We might see the same effect if a proportion of the 
weaker students are leaving along the way” (p. 20). Facione went on the state: “The 
definitive analysis needed to determine gains over the course of a nursing program is 
paired t-test of the pre-test and post-test scores for the same individuals” (p. 20). 

Change in Individual Thinking Skills 

To assess the change in thinking skills, the pre-test CCTST scores were compared 
to the post-test CCTST scores. The comparison was made by subtracting the pre-test 
score from the post-test score for each individual. The resulting change scores were 
analyzed with the t-test of differences. 

Sample A student scores showed a change in total thinking skills of 0.900 
between pre- and post-test scores on the CCTST. Their t-test of differences t-value for 
total thinking skills was 0.93. This t-value is not significant at the .05 level. In other 
words, there is no statistically significant gain in thinking skills during the program as 
measured by the total score. Sample A student scores showed a change in Analysis 
thinking skills of -0. 150 between pre- and post-test scores on the CCTST. This indicates 
a decrease in analysis thinking skills. This was not significant. Student scores for sample 
A showed a change in Evaluation thinking skills of 1.450 between pre- and post-test 
scores on the CCTST. Their t-test of differences t value for Evaluation thinking skills 
was 2.29. This was significant at the .05 level. In other words, this indicates a 
significant increase in evaluation thinking skills for Sample A. Sample A student scores 
showed a change in Inference thinking skills of -0.550 between pre- and post-test scores 
on the CCTST. This was a decrease in Inference thinking skills. This was not 
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significant. Sample A student scores showed a change in Deductive Reasoning skills of 
0.900 between pre- and post-test scores on the CCTST. Their t-test of differences t value 
for Deductive Reasoning was 1.66. This change was not significant. Sample A student 
scores showed no change in Inductive Reasoning skills. (See Table 1 1 : Sample A 
Change in Critical Thinking Test Scores). 

Table 1 1 

Sample A Change in Critical Thinking Test Scores 







Change in Critical Thinking 




Test Area 


N 


Mean 

Change 


Standard 

Deviation 


t 


P 


Total CCTST 


20 


0.900 


4.315 


0.93 


0.36 


Analysis Subset 


20 


-0.150 


1.531 


-0.44 


0.67 


Evaluation Subset 


20 


1.450 


2.837 


2.29 


0.034 


Inference Subset 


20 


-0.550 


2.305 


-1.07 


0.30 


Deductive Reasoning Subset 


20 


0.900 


2.426 


1.66 


0.11 


Inductive Reasoning Subset 


20 


No change 


3.009 


0.00 


1.00 



Sample B student scores showed a change in total thinking skills of 1.304 
between pre- and post-test scores on the CCTST. Their t-test of differences t value for 
total thinking skills was 2. 17. This was significant at the .05 level. In other words, this 
indicates a significant increase in total thinking skills for Sample B. Sample B showed a 
change in Analysis thinking skills of 0.390 between pre- and post-test scores on the 
CCTST. This indicates a minimal change in analysis thinking skills. This was not 
significant. Sample B student scores showed a change in Evaluation thinking skills of 
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0^52 between pre- and post-test scores on the CCTST. Their t-test of differences (t) 
scores for total thinking was 1.71. This change was not significant. Sample B student 
scores showed a change in Inference thinking skills of 0.217 between pre- and post-test 
scores on the CCTST. Their t-test of differences t value for Inference thinking was 0.49. 
This change was not significant. Sample B student scores showed a change in Deductive 
Reasoning skills of 0.087 between pre- and post-test scores on the CCTST. Their t-test 
of differences t values for Deductive Reasoning 0. 16. This was not a significant change. 
Sample B student scores showed a change in Inductive Reasoning skills of 0.826 between 
pre- and post-test scores on the CCTST. Their t-test of differences t values for Inductive 
Reasoning 2.30. This was significant at the .05 level. In other words, this indicates a 
significant increase in inductive reasoning skills for Sample B. (See Table 12: Sample B 
Change in Critical Thinking Test Scores). 

Table 12 

Sample B Change in Critical Thinking Test Scores 



Change in Critical Thinking 



Test 


N 


Mean 


Standard 

Deviation 


t 


P 


Total CCTST 


23 


1.304 


2.883 


2.17 


0.041 


Analysis Subset 


23 


0.390 


1.588 


1.18 


0.25 


Evaluation Subset 


23 


0.652 


1.824 


1.71 


0.10 


Inference Subset 


23 


0.217 


2.110 


0.49 


0.63 


Deductive Reasoning Subset 


23 


0.087 


2.610 


0.16 


0.87 


Inductive Reasoning Subset 


23 


0.826 


1.723 


2.30 


0.031 
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The change score results for both Sample A and Sample B are higher than those 
reported for BSN programs. Facione (1997) reported the change in scores for 625 
students with pre-test and post-test scores. The post-test mean for BSN students was 0.68 
higher than pre-test mean. Facione stated; “The gain of nearly one question, represents 
meaningful movement when one considers the difficulty of the instrument and the effort 
involved in pushing a 625 undergraduate student mountain one click in the positive 
direction” (p. 25). In comparison, MATC’s total thinking mean showed a change of .90 
for Sample A and 1.304 for Sample B. 

Correlations between ACT, GPA, and Thinking Test Scores 

Pearson s product moment correlation was used to measure the correlations 
between ACT, Grade Point Average (GPA), pre-test, post-test, and pre-/post-tests. ACT 
scores, with one exception, had a higher correlation with critical thinking skills than 
cumulative grade point averages. In addition, the correlations were moderate (.4 to .6) to 
low (.2 to .39). (See Table 13: Correlations). 

The mean CCTST pre-test scores of Sample A were compared to the mean 
CCTST pre-test scores of Sample B using a two sample t-test. Pre-test thinking skills 
scores were almost identical for the two groups. The t value was -0.01 and was not 
significant. (See Table 14: Correlations of Pre-test and Post-test Scores). 

The mean CCTST post-test scores of Sample A were compared to the mean 
CCTST post-test scores of Sample B using a two sample t-test. There was no significant 
difference in post-test thinking skills scores between the two groups. (See Table 14: 
Correlation of Pre-test and Post-test Scores). 
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Table 13 
Correlations 



Sample A Sample B 





GPA 


ACT 


GPA 


ACT 


GPA 




0.359 




0.222 


Pre-test 


0.275 


0.558* 


0.398* 


0.408* 


Post-test 


0.603* 


0.474* 


0.298 


0.455* 


Pre-/Post-test 


0.389* 


0.213 


0.068 


0.325 



* A correlation coefficient of .36 or larger is significant at the .05 level. Another words, 
there is a significant relationship between the two values. 



Pre-test and post-test scores for Sample A and Sample B were compared. The 
comparison were between groups with two-sample t-tests to determine if there were any 
significant starting and ending differences. Sample A had a mean change of +.9 for total 
thinking skills and Sample B had a mean change of + 1 .304 for total thinking skills. (See 
Table 14; Comparisons of Pre-test and Post-test Scores). 

Table 14 

Comparisons of Pre-test and Post-test Scores 



Sample A Sample B 





N 


Mean 


N 


Mean 


t-test 


Probability 


Pre-test Scores 


20 


14.95 


23 


14.96 


-0.01 


0.99 


Post-test scores 


20 


15.85 


23 


16.26 


-0.26 


0.99 



Sample A had a mean change of -. 15 for Analysis thinking skills; Sample B had a mean 
change of +.39 for Analysis thinking skills. Sample A had a mean change of + 1 .45 for 
Evaluation thinking skills; Sample B had a mean change of +.65 for Evaluation thinking 
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skills. Sample A had a mean change of -.55 for Inference thinking skills; Sample B had 
a mean change of +.217 for Inference thinking skills. Sample A had a mean change of 
+.9 for Deductive Reasoning skills; Sample B had a mean change of +.087 for Deductive 
Reasoning skills. Sample A had no change for Inductive Reasoning skills; Sample B had 
a mean change of +.826 for Inductive Reasoning skills. (See Figure A; Comparisons of 
Change in Thinking Skills). 

Figure A 

Comparisons of Change in Thinking Skills 



Comparisons of Change in Thinking 

Skills 




Sample A had a significant increase in Evaluation thinking skills. Sample B had 
a significant increase in Total thinking skills. A significant increase was also shown in 
the subset of Inductive Reasoning. Sample B had positive gains in all subset thinking 
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tests, while Sample A had decreases in two areas of thinking tests (Analysis and 
Inference). (See Table 15; Analysis of Thinking Skills Scores). 

Table 15 

Analysis of Thinking Skills Change Scores 







Sample A 




Sample B 


Test Change Scores 




Mean P 




Mean P 


Total CCTST Score 




+.9 0.36 




+1.304 0.041 


Analysis Subset 




-.15 0.67 




+.39 0.25 


Evaluation Subset 




+1.45 0.034 




+.65 0.10 


Inference Subset 




-.55 0.30 




+.217 0.63 


Deductive Reasoning Subset 




+.9 0.11 




+.087 0.87 


Inductive Reasoning Subset 




0.00 1.00 




+.826 0.031 


Table 16 










Significan^ej)f Thinking Skills Scores 










Sample A 


Sample B 


Test 


Mean 


Significance 


Mean 


Significance 


Total CCTST Score 


+.9 


Not Sig 


+1.304 


Significant 

Increase 


Analysis Subset 


-.15 


Not Sig 


+.39 


Not Sig 


Evaluation Subset 


+1.45 


Significant 

Increase 


+.65 


Not Sig 


Inference Subset 


-.55 


Not Sig 


+.217 


Not Sig 


Deductive Reasoning Subset 


+.9 


Not Sig 


+.087 


Not Sig 


Inductive Reasoning Subset 


0.00 


Not Sig 


+.826 


Significant 

Increase 
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The results of the degree of difference in the sub-sets CCTST mean post-test 
scores and the pre-test scores in MATC ADN students were significant in the sub-sets 
Evaluation and Inductive Reasoning. Facione (1997, p. 27) also reports gains in the sub- 
sets Evaluation and Inductive Reasoning for 625 BSN students. 

Summary 

Neither group had significant changes in the areas of Analysis, Inference, or 
Deductive thinking skills. Group A had a significant increase in Evaluation skills. 

Group B had a significant increase in total scores and Inductive Reasoning. In addition. 
Group B had positive gains in all of the sub-tests. The significance level used was .05. 
(See Table 16: Significance of Thinking Skills Scores). 
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Summary, Conclusions, and Recommendations 

Introduction 

The study was a comparative study of two groups of Associate Degree Nursing 
students at Madison Area Technical College - Reedsburg. The study focused on the 
change in critical thinking skills as they progressed through their educational process at 
Madison Area Technical College - Reedsburg. 

Summary 

Critical thinking is a major component of nursing education. Programs in 
nursing must assess critical thinking as a required outcome for national accreditation by 
the National League for Nursing. In March, 1996, Madison Area Technical College 
Associate Degree Nursing faculty identified critical thinking as a major content thread to 
be carried as a priority for instruction through all four semesters of the Associate Degree 
Nursing program. 

The purpose of this study was to determine the degree of change in critical 
thinking skills in Associate Degree Nursing students as they progress through their 
educational process at Madison Area Technical College. Cohorts of twenty-four 
students, entering the Associate Degree Nursing program at Madison Area Technical 
College - Reedsburg campus, in August, 1995 and August, 1996 and graduating in May, 
1997 and May, 1998, comprised the samples. The California Critical Thinking Skills 
Test - Form A (CCTST) was administered to the entering Associate Degree Nursing class 
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in August of 1995 and in August of 1996 - prior to the beginning of program instruction. 
The California Critical Thinking Skills Test Form A was also administered at the 
completion of the program of study (May, 1997 and May, 1998). The California Critical 
Thinking Skills Test - Form A was administered by the researcher at the Madison Area 
Technical College - Reedsburg campus. 

To assess the degree of change in thinking skills, the pre-test CCTST scores were 
compared to the post-test CCTST scores. The comparison was made by subtracting the 
pre-test score from the post-test score for each individual. The resulting change scores 
were analyzed with the t-test of differences. 

The major outcome of this study was to identify CCTST pre- and post-testing of 
MATC’s ADN students as a viable means of assessing their change in critical thinking 
skills. One group showed a significant increase in the total score for the CCTST, based 
on positive gains in their scores for the five sub-tests that impacted the total score. 

Further outcomes were to identify a significant change in both evaluation and inductive 
reasoning thinking skills in the two groups of students: One group showed a significant 
increase in evaluation thinking skills; the other group had a positive gain in evaluation 
thinking skills, although it was not statistically significant. One group showed a 
significant increase in the critical thinking skill subset Inductive Reasoning. An 
additional benefit of the study was to identify limited, if any, change in analysis, 
inference, and deductive thinking skills in the program students. These areas of limited 
change then become areas to be further evaluated by the ADN program. 
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Conclusions 

The conclusions of this study were based on the following research hypotheses. 

1 . There will be an increase in the critical thinking skills of Associate 
Degree Nursing students from entry to completion of the Associate 
Degree Nursing program at Madison Area Technical College - Reedsburg. 
Sample B showed a significant increase in the total score for the CCTST. 
This was based on positive gains in their scores for the five sub-tests that 
impacted the total. 

2. There will be an increase in the critical thinking skill subset “Analysis” in 
Associate Degree Nursing students from entry to completion of the 
Associate Degree Nursing program at Madison Area Technical College - 
Reedsburg. There was no significant change in the critical thinking skill 
subset Analysis for either group. 

3. There will be an increase in the critical thinking skill subset “Evaluation” 
in Associate Degree Nursing students from entry to completion of the 
Associate Degree Nursing program at Madison Area Technical College- 
Reedsburg. Sample A showed a significant increase in the critical 
thinking skill subset Evaluation. Although the gain was not statistically 
significant. Sample B did have a positive gain in Evaluation thinking 
skills. 

4. There will be an increase in the critical thinking skill subset “Inference” 
in Associate Degree Nursing students from entry to completion of the 
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Associate Degree Nursing program at Madison Area Technical College - 
Reedsburg. There was no significant change in the critical thinking skill 
subset Inference for either group. 

5. There will be an increase in the critical thinking skill subset “Deductive 
Reasoning” in Associate Degree Nursing students from entry to 
completion of the Associate Degree Nursing program at Madison Area 
Technical College- Reedsburg. There was no significant change in the 
critical thinking skill subset Deductive Reasoning for either group. 

6. There will be an increase in the critical thinking skill subset “Inductive 
Reasoning” in Associate Degree Nursing students from entry to 
completion of the Associate Degree Nursing program at Madison Area 
Techmcal College- Reedsburg. Sample B showed a significant increase in 
the critical thinking skill subset Inductive Reasoning. 

Further conclusions of this study were based on the following research objectives. 
Research Objective Number 1 

Research Objective Number 1 was to determine the degree of change in critical 
thinking skills in Associate Degree Nursing students at Madison Area Technical College 
- Reedsburg. This research showed mixed results. One group had significant change in 
total critical thinking skills. The other group had a positive change in total thinking 
skills, but it was not statistically significant. However, both groups showed significant 
change in evaluation thinking skills, and inductive reasoning skills. The comparison was 
made by subtracting the pre-test score from the post-test score for each individual. The 
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resulting change scores were analyzed with the t-test of differences. 

Research Objective Number 2 

Research Objective Number 2 was to assess the feasibility of using the California 
C. Ptical Thinking Skills Test Form A (CCTST) to develop performance standards related 
to critical thinking for Madison Area Technical College Associate Degree Nursing 
program. It is feasible to use the CCTST to assess performance standards related to 
critical thinking for the Nladison Area Technical College Associate Degree Nursing 
program. The CCTST has been identified by the National League for Nursing as a 
means of assessment of change in critical thinking skills. The CCTST is a standardized 
test, which can be made available throughout the district. The length of the test fits 
within the constraints of class periods. The placements of the tests at the beginning and 
end of the program fit within program schedules and class timetables. The directions to 
test administrators and to students are easily understood and implemented. The 
conditions for testing are easily met within the Madison Area Technical College campus 
environments. 

The concept of the assessment of critical thinking in nursing students is new to 
the national nursing accreditation process. ADN programs throughout Wisconsin and the 
nation are struggling with the need to validate a change in critical thinking skills in 
nursing students. Assessment of critical thinking skills is to be the focus of the annual 
Associate Degree Nursing program statewide conference being held April, 1999. 

Meanwhile, the MATC ADN program is in the process of reaching a consensus 
definition for critical thinking (the first step for the accreditation process) and then in 
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completing curriculum revision and implementation, which includes instructional 
methods to promote critical thinking. The ADN program curriculum committee has 
identified the need to assist all program faculty to better understand how to implement 
critical thinking as a priority for the 1999 - 2000 school year. 

Recommendations 

The following recommendations are made based on the results of this study: 

1 . Due to the conclusions reached by this study, it is recommended that the 
CCTST be administered as a pre-test and post-test to all students in the 
Associate Degree Nursing program at Madison Area Technical College. 

2. It is recommended that Madison Area Technical College increase its 
curriculum to strengthen Analysis, Inference, and Deductive Reasoning 
thinking skills. 

3. Research should be continued with further cohorts of students in the 
Associate Degree Nursing program at Madison Area Technical College. 
Further longitudinal studies need to be conducted to determine the degree 
of difference in critical thinking skills between students in the old 
curriculum and the students in the new curriculum. This continued 
research gains importance since the program curriculum had significant 
changes beginning in August, 1997 to include Critical Thinking as a 
continuing thread throughout all four semesters of the program. 

Significant curricular changes have been made to implement critical 
thinking. 
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Recomm endations for Additional Research 

The following recommendations would be of further benefit to Associate Degree 
Nursing education: 

1. Formal studies are needed to develop a standard for critical thinking skills for 
Associate Degree Nursing students and Associate Degree Nursing programs. In 1997, the 
California Critical Thinking Skills Test was identified by the National League of Nursing 
as an appropriate instrument for assessment of critical thinking skills. Facione (1997) 
reported upon a longitudinal aggregate data analysis of assessment of critical thinking 
skills in baccalaureate and graduate nursing students, using the California Critical 
Thinking Skills Test. Associate Degree Nursing students were not included in this study. 
Assistance is needed for Associate Degree Nursing programs to set standards related to 
critical thinking outcomes for their educational programs. 

2. Formal studies are needed to test the hypothesis that what nursing faculty is 
doing in relation to critical thinking and the curriculum will have an effect on students’ 
success in Associate Degree Nursing programs. Activities of programs or faculty in 
relation to critical thinking might include: 

Adopting a critical thinking focus for the program. 

Discussing with faculty members the meaning of critical thinking. 

Planning to make curricular changes to enhance critical thinking. 

Implementing curricular changes. 

Requiring a designated critical thinking course. 

Analyzing critical thinking test score data relative to critical thinking outcomes. 
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Interpreting critical thinking test score data for purpwses of teaching or curricular 

development. (Facione, 1997, p. 42). 

Facione (1997) has conducted extensive research related to the effect of faculty 
and program involvement with critical thinking for baccalaureate and graduate nursing 
student success. However, little research has been conducted which includes Associate 
Degree Nursing programs. 

3. Further research needs to conducted to determine the correlation between 
successful completion of the National Licensure Examination for Registered Nurses and 
critical thinking ability. 

4. Further research needs to be undertaken to assess the affect of student 
motivation, placement of testing, students’ regard for faculty, and students’ 
understanding of the application of outcomes upon the pre-test and post-test scores. 
Students exiting the program may in particular give less than their best effort on an 
ungraded test and instead be focused upon their professional future and specifically in a 
job placement. 

5. It would be useful to conduct a concurrent validity check on the critical 
thinking skills test for the MATC ADN program. For example, assessment 
situations/simulations for nursing skills/nursing cares which evaluate critical thinking 
skills need to be developed for the program. Then, a comparison could be made between 
student’s performance in these situations and their CCTST scores. 
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The following are summaries of reviews of the California Critical Thinking Skills 
Test in further support of its validity as an assessment of critical thinking. First of all, a 
review of the California Critical Thinking Skills Test by Robert F. McMorris, Professor 
of Educational Psychology and Statistics, State University of New York at Albany, 
Albany, New York: 

In Roget’s Thesaurus (Chapman, 1992, p. 842), a synonym given for “critical” is 
“crucial.” Certainly critical thinking skills are crucial for individuals and society, 
and developing an appropriate test to assess critical thinking skills is no trivial task. 
The California Critical Thinking Skills Test (CCTST) contains 34 multiple-choice 
items with a 45 minute time limit. The items cover a variety of topics: Some items 
are realistic, providing high face validity, but they potentially confound reasoning 
with content; and some are “nonsense,” content-free items for those who prefer a 
more abstract approach. The items seem reasonably interesting. 

A Delphi panel developed a consensus definition of critical thinking (CT), core CT 
skills with examples, and dispositions crucial to becoming a critical thinker. The test 
contains three subscores based on the panel’s work: Analysis, Evaluation, and 
Inference. The developer, Facione, also apologetically offers two other subscores. 
Deductive Reasoning and Inductive Reasoning, based on 30 of the 34 items, to meet 
a California State University objective. (Given the double use of most items, we will 
refer to subscores rather than subtests.) 

The developer conducted several studies using the CCTST at California State in 
Fullerton involving 1,196 undergraduates, 20 instructors, 5 different courses, and 
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three academic departments. These data are bases for validity, reliability, and norm 
information. 

Validity. Test users are urged to study the items to judge validity, especially to 
estimate whether the tests meets their conceptualizations of critical thinking. 
(Conoley & Impara, 1994, p. 37 - 39). 

According to the CCTST Fact Sheet (Facione, 1990), “The CCTST measures the 
growth in CT skills which is an intended outcome of completing a college level general 
education CT course. Facione has collected considerable pre and post data, some quite 

cleverly by measuring post at the end of the first semester and pre with similar students 
at the beginning of the second semester. 

Gam runs between .04 and 1.45 in mean scores on a 34-item test, statistically 
significant with large samples. 

Pretest scores correlate with college GPA (.20), SAT-V (.55), SAT-M (.44), most 
Nelson-Denny Reading scores (.40s), and post-test scores (.70). In general, these 

correlations with tests and with grades seem reasonable and supportive. (Facione, 
1990, p. 8). 

Facione may emphasize statistics instead of meanfulness in examining gender 
differences ... Yet the mean difference pre and post are almost equal and hardly 
impressive (about 3/4 point). The difference in sample size - 479 [students] for pre and 
710 [students] post - helps in estimating why the posttests differ statistically by gender. 

Another way to consider validity is through multi trait-multi method matrices 
(Cambell & Fiske, 1959). The author sets the stage; “Three [subscores] ...’Analysis,’ 
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Evaluation, and Inference, correlate strongly with each other and with the overall 
CCTST...same is true of the two CCTST [subscores], ‘Deductive Reasoning’ and 
‘Inductive Reasoning,’ which divide CCTST items along the more traditional matrix.” 
Treating pre and post as methods and the five subscores as traits, the validity diagonal 
contained one correlation coefficient in the ,30s, two in the ,40s, and two in the ,50s. 

The hetero trait-monomethod triangles contained two correlations in the ,20s, three in the 
•30s, and three in the .40s. Three estimates of reliability were provided for the total 
score: KR20s for pretest (.69) and post-test (.68), and a pre-post correlation (.70). 
Subscore reliability might be estimated using Spearman-Brown; starting from .70 for the 
total score, reliability for a half-length test would be approximately .54. Given the 
estimate of reliability, the validity coefficients among subscores appear to this reviewer 
as reasonably supportive. 

Reliability. To summarize the information given above, total-score internal 
consistency appears to be close to .70. Subscore reliability, although not provided, 
might be in the .50s. No standard errors of measurement are provided; for the total 
score the standard error is about 2.5 based on either the typical formula or Lord’s 
approximation. 

The reliability information does not support interpretation of differences for 
individuals, either for a profile or for gain. Subscore reliability is too low and 
intercorrelations too high to allow a profile to be dependable. Assessment of gain 
appears impossible: With total KR20s of .69 and .69, and pre-post correlation of .70, 
the reliability of the difference is estimated to be 0. 
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Norms. As noted above, the norm group is composed of undergraduates from one of 
the California State col leges... The groups contain both native and nonnative English 
speakers; subgroup norms could easily have been provided. Percentiles are provided 
for both total score and the five subscores (Conoley & Impara, 1994, p. 37 . 39 ). 

William B. Michael, Professor of Education and Psychology, University of Southern 
California, Los Angeles, California, adds the following: 

The CCTST would appear to possess substantial content validity - perhaps more than 
any other competing instrument in light of the collective wisdom of the eminent 
scholars who contributed to its development. The resulting score distributions, which 
are normal m form, provide a basis for differentiating quite adequately among the 
examinees. The 34 items are not easy. On the pretest the highest score for the 
sample of students studied was 29, and on the post-test 3 1 with a respective means of 

15.89 ad 17.27. There may be some concern relative to the 45-minute time limit 
(Conoley & Impara, 1994, p. 39 - 40). 
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Agreement to Participate as a Research Subject 

Project Title: T^he Degree of Change in Critical Thinking Skills in Associate Degree Nursine 
Students ® 

This research examines the ability to use critical thinking skills. The practice of professional 
nursing requires the use of critical thinking skills. Critical thinking is a major aim of nursing 
education_ The Madison Area Technical College Associate Degree Nursing Advisory committee 
has Identified critical thinking as an essential skill for nursing graduates. This study^ll 
Fo^^ thinking ability over the two years of study in the Associate Degree Nursing 

RISKS: 

There is little or no risk to you in taking the tests. The California Critical Thinking Skills Test is 

th^kinr' 

BENEFITS: 

'"Tu" in the future 

there IS no direct benefit to you by participating in this study. MATC has always been very 

sTildent^ program. You will help to maintain that quality for future program 
CONFIDENTIALITY OF RESPONSES: 

Your answers are strictly confidential. Only the primary researcher or her designee will have 
access to the confidential raw data. Neither the answers or results of this study will in any wav 
affect your course grades or your progress through this program of study. Neither your name 
identiiying number, or you personal answers will ever be published. 

RIGHT TO WITHDRAW OR DECLINE TO PARTICIPATE: 

Your participation in this study is entirely voluntary. You may choose not to participate without 

participate and later wish to withdraw 

from the study, you may discontinue your participation at that time without incurring adverse 
consequences. ° 



r anest that I have read and understand the above descriptions, including potential risks, benelits 
and my rights as a participant, and that all my questions about the study have been answerrd to 
my sattslaction. I hereby give my informed consent to participate in <L research smT 

Signature 
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